[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"summary-30072e6e8b386729-mrc-openai-s-protocol-for-resilient-ai-training-ne-summary":3,"summaries-facets-categories":95,"summary-related-30072e6e8b386729-mrc-openai-s-protocol-for-resilient-ai-training-ne-summary":3680},{"id":4,"title":5,"ai":6,"body":13,"categories":55,"created_at":57,"date_modified":57,"description":49,"extension":58,"faq":57,"featured":59,"kicker_label":57,"meta":60,"navigation":77,"path":78,"published_at":79,"question":57,"scraped_at":80,"seo":81,"sitemap":82,"source_id":83,"source_name":84,"source_type":85,"source_url":86,"stem":87,"tags":88,"thumbnail_url":57,"tldr":92,"tweet":57,"unknown_tags":93,"__hash__":94},"summaries\u002Fsummaries\u002F30072e6e8b386729-mrc-openai-s-protocol-for-resilient-ai-training-ne-summary.md","MRC: OpenAI's Protocol for Resilient AI Training Networks",{"provider":7,"model":8,"input_tokens":9,"output_tokens":10,"processing_time_ms":11,"cost_usd":12},"openrouter","x-ai\u002Fgrok-4.1-fast",8465,1915,20569,0.00214365,{"type":14,"value":15,"toc":48},"minimark",[16,21,25,28,31,35,38,41,45],[17,18,20],"h2",{"id":19},"multipath-mechanisms-eliminate-congestion-and-enable-fast-recovery","Multipath Mechanisms Eliminate Congestion and Enable Fast Recovery",[22,23,24],"p",{},"In large AI training clusters, network congestion, link failures, and jitter cause GPU idle time, amplifying costs as clusters scale to millions of data transfers per step. MRC builds on RoCEv2 for hardware-accelerated RDMA over Ethernet and SRv6 for static source routing, shifting intelligence to NICs while switches follow pre-configured paths blindly. This avoids interference from dynamic routing.",[22,26,27],{},"Adaptive packet spraying distributes packets across hundreds of paths at the NIC level, achieving higher bandwidth, reduced tail latency, and packet-level load balancing—unlike single-path RoCEv2. For failures, MRC detects issues in microseconds and reroutes: if an 8-port 800Gb\u002Fs NIC loses one port, it drops to 7\u002F8 capacity but recalculates paths instantly, notifies peers to avoid the failed plane, and restores it within a minute upon recovery. Conventional fabrics take seconds to tens of seconds, often crashing jobs; MRC keeps training alive with minimal performance hit.",[22,29,30],{},"AMD's NSCC congestion control integrates via UEC specs, preserving RDMA semantics while adding multipath support.",[17,32,34],{"id":33},"multi-plane-architecture-cuts-tiers-costs-and-latency","Multi-Plane Architecture Cuts Tiers, Costs, and Latency",[22,36,37],{},"MRC reimagines NICs as multiple smaller links (e.g., one 800Gb\u002Fs interface split into eight 100Gb\u002Fs to eight switches), enabling a two-tier Clos network for 131,000 GPUs versus three-to-four tiers in 800Gb\u002Fs designs. Longest paths cross three switches instead of five-to-seven, slashing latency.",[22,39,40],{},"For full bisection bandwidth, this uses 2\u002F3 the optics and 3\u002F5 the switches of three-tier networks, reducing power, cost, and failure blast radius. A tier-1 switch failure (e.g., rebooting four during training) no longer halts jobs.",[17,42,44],{"id":43},"production-on-named-hardware-across-openai-clusters","Production on Named Hardware Across OpenAI Clusters",[22,46,47],{},"Deployed on 400\u002F800Gb\u002Fs RDMA NICs like NVIDIA ConnectX-8, AMD Pollara\u002FVulcano, Broadcom Thor Ultra; SRv6 switches include NVIDIA Spectrum-4\u002F5 (Cumulus\u002FSONiC) and Broadcom Tomahawk 5 (Arista EOS). Powers NVIDIA GB200 supercomputers in OpenAI's Stargate (OCI Abilene, TX) and Microsoft's Fairwater (Atlanta\u002FWisconsin), training ChatGPT and Codex models without job interruptions from failures.",{"title":49,"searchDepth":50,"depth":50,"links":51},"",2,[52,53,54],{"id":19,"depth":50,"text":20},{"id":33,"depth":50,"text":34},{"id":43,"depth":50,"text":44},[56],"DevOps & Cloud",null,"md",false,{"content_references":61,"triage":72},[62,67],{"type":63,"title":64,"url":65,"context":66},"paper","Resilient AI Supercomputer Networking using MRC and SRv6","https:\u002F\u002Fcdn.openai.com\u002Fpdf\u002Fresilient-ai-supercomputer-networking-using-mrc-and-srv6.pdf","cited",{"type":68,"title":69,"url":70,"context":71},"other","MRC Supercomputer Networking Technical Details","https:\u002F\u002Fopenai.com\u002Findex\u002Fmrc-supercomputer-networking\u002F","recommended",{"relevance":73,"novelty":73,"quality":74,"actionability":50,"composite":75,"reasoning":76},3,4,3.05,"Category: AI & LLMs. The article discusses OpenAI's MRC protocol, which is relevant to AI infrastructure but lacks direct applicability for product builders looking for actionable insights. While it presents some new technical details about network optimization for AI training, it does not provide practical steps or frameworks that the audience can implement.",true,"\u002Fsummaries\u002F30072e6e8b386729-mrc-openai-s-protocol-for-resilient-ai-training-ne-summary","2026-05-07 07:50:02","2026-05-07 11:24:11",{"title":5,"description":49},{"loc":78},"30072e6e8b386729","MarkTechPost","article","https:\u002F\u002Fwww.marktechpost.com\u002F2026\u002F05\u002F07\u002Fopenai-introduces-mrc-multipath-reliable-connection-a-new-open-networking-protocol-for-large-scale-ai-supercomputer-training-clusters\u002F","summaries\u002F30072e6e8b386729-mrc-openai-s-protocol-for-resilient-ai-training-ne-summary",[89,90,91],"machine-learning","devops","cloud","OpenAI's MRC extends RoCE with multipath spraying, microsecond failure recovery via SRv6, and multi-plane designs to deliver predictable performance in 131k-GPU clusters, using 2\u002F3 fewer optics and 3\u002F5 fewer switches than traditional setups.",[],"XbDsma4E_5cuB3WLtPi6GgqSNlQtb2CdSK-eHkIrlrc",[96,99,102,105,108,111,113,115,117,119,121,123,126,128,130,132,134,136,138,140,142,144,147,150,152,154,157,159,161,164,166,168,170,172,174,176,178,180,182,184,186,188,190,192,194,196,198,200,202,204,206,208,210,212,214,216,218,220,222,224,226,228,230,232,234,236,238,240,242,244,246,248,250,252,254,256,258,260,262,264,266,268,270,272,274,276,278,280,282,284,286,288,290,292,294,296,298,300,302,304,306,308,310,312,314,316,318,320,322,324,326,328,330,332,334,336,338,340,342,344,346,348,350,352,354,356,358,360,362,364,366,368,370,372,374,376,378,380,382,384,386,388,390,392,394,396,398,400,402,404,406,408,410,412,414,416,418,420,422,424,426,428,430,432,434,436,438,440,442,444,446,448,450,452,454,456,458,460,462,464,466,468,470,472,474,476,478,480,482,484,486,488,490,492,494,496,498,500,502,504,506,508,510,512,514,516,518,520,522,524,526,528,530,532,534,536,538,540,542,544,546,548,550,552,554,556,558,560,562,564,566,568,570,572,574,576,578,580,582,584,586,588,590,592,594,596,598,600,602,604,606,608,610,612,614,616,618,620,622,624,626,628,630,632,634,636,638,640,642,644,646,648,650,652,654,656,658,660,662,664,666,668,670,672,674,676,678,680,682,684,686,688,690,692,694,696,698,700,702,704,706,708,710,712,714,716,718,720,722,724,726,728,730,732,734,736,738,740,742,744,746,748,750,752,754,756,758,760,762,764,766,768,770,772,774,776,778,780,782,784,786,788,790,792,794,796,798,800,802,804,806,808,810,812,814,816,818,820,822,824,826,828,830,832,834,836,838,840,842,844,846,848,850,852,854,856,858,860,862,864,866,868,870,872,874,876,878,880,882,884,886,888,890,892,894,896,898,900,902,904,906,908,910,912,914,916,918,920,922,924,926,928,930,932,934,936,938,940,942,944,946,948,950,952,954,956,958,960,962,964,966,968,970,972,974,976,978,980,982,984,986,988,990,992,994,996,998,1000,1002,1004,1006,1008,1010,1012,1014,1016,1018,1020,1022,1024,1026,1028,1030,1032,1034,1036,1038,1040,1042,1044,1046,1048,1050,1052,1054,1056,1058,1060,1062,1064,1066,1068,1070,1072,1074,1076,1078,1080,1082,1084,1086,1088,1090,1092,1094,1096,1098,1100,1102,1104,1106,1108,1110,1112,1114,1116,1118,1120,1122,1124,1126,1128,1130,1132,1134,1136,1138,1140,1142,1144,1146,1148,1150,1152,1154,1156,1158,1160,1162,1164,1166,1168,1170,1172,1174,1176,1178,1180,1182,1184,1186,1188,1190,1192,1194,1196,1198,1200,1202,1204,1206,1208,1210,1212,1214,1216,1218,1220,1222,1224,1226,1228,1230,1232,1234,1236,1238,1240,1242,1244,1246,1248,1250,1252,1254,1256,1258,1260,1262,1264,1266,1268,1270,1272,1274,1276,1278,1280,1282,1284,1286,1288,1290,1292,1294,1296,1298,1300,1302,1304,1306,1308,1310,1312,1314,1316,1318,1320,1322,1324,1326,1328,1330,1332,1334,1336,1338,1340,1342,1344,1346,1348,1350,1352,1354,1356,1358,1360,1362,1364,1366,1368,1370,1372,1374,1376,1378,1380,1382,1384,1386,1388,1390,1392,1394,1396,1398,1400,1402,1404,1406,1408,1410,1412,1414,1416,1418,1420,1422,1424,1426,1428,1430,1432,1434,1436,1438,1440,1442,1444,1446,1448,1450,1452,1454,1456,1458,1460,1462,1464,1466,1468,1470,1472,1474,1476,1478,1480,1482,1484,1486,1488,1490,1492,1494,1496,1498,1500,1502,1504,1506,1508,1510,1512,1514,1516,1518,1520,1522,1524,1526,1528,1530,1532,1534,1536,1538,1540,1542,1544,1546,1548,1550,1552,1554,1556,1558,1560,1562,1564,1566,1568,1570,1572,1574,1576,1578,1580,1582,1584,1586,1588,1590,1592,1594,1596,1598,1600,1602,1604,1606,1608,1610,1612,1614,1616,1618,1620,1622,1624,1626,1628,1630,1632,1634,1636,1638,1640,1642,1644,1646,1648,1650,1652,1654,1656,1658,1660,1662,1664,1666,1668,1670,1672,1674,1676,1678,1680,1682,1684,1686,1688,1690,1692,1694,1696,1698,1700,1702,1704,1706,1708,1710,1712,1714,1716,1718,1720,1722,1724,1726,1728,1730,1732,1734,1736,1738,1740,1742,1744,1746,1748,1750,1752,1754,1756,1758,1760,1762,1764,1766,1768,1770,1772,1774,1776,1778,1780,1782,1784,1786,1788,1790,1792,1794,1796,1798,1800,1802,1804,1806,1808,1810,1812,1814,1816,1818,1820,1822,1824,1826,1828,1830,1832,1834,1836,1838,1840,1842,1844,1846,1848,1850,1852,1854,1856,1858,1860,1862,1864,1866,1868,1870,1872,1874,1876,1878,1880,1882,1884,1886,1888,1890,1892,1894,1896,1898,1900,1902,1904,1906,1908,1910,1912,1914,1916,1918,1920,1922,1924,1926,1928,1930,1932,1934,1936,1938,1940,1942,1944,1946,1948,1950,1952,1954,1956,1958,1960,1962,1964,1966,1968,1970,1972,1974,1976,1978,1980,1982,1984,1986,1988,1990,1992,1994,1996,1998,2000,2002,2004,2006,2008,2010,2012,2014,2016,2018,2020,2022,2024,2026,2028,2030,2032,2034,2036,2038,2040,2042,2044,2046,2048,2050,2052,2054,2056,2058,2060,2062,2064,2066,2068,2070,2072,2074,2076,2078,2080,2082,2084,2086,2088,2090,2092,2094,2096,2098,2100,2102,2104,2106,2108,2110,2112,2114,2116,2118,2120,2122,2124,2126,2128,2130,2132,2134,2136,2138,2140,2142,2144,2146,2148,2150,2152,2154,2156,2158,2160,2162,2164,2166,2168,2170,2172,2174,2176,2178,2180,2182,2184,2186,2188,2190,2192,2194,2196,2198,2200,2202,2204,2206,2208,2210,2212,2214,2216,2218,2220,2222,2224,2226,2228,2230,2232,2234,2236,2238,2240,2242,2244,2246,2248,2250,2252,2254,2256,2258,2260,2262,2264,2266,2268,2270,2272,2274,2276,2278,2280,2282,2284,2286,2288,2290,2292,2294,2296,2298,2300,2302,2304,2306,2308,2310,2312,2314,2316,2318,2320,2322,2324,2326,2328,2330,2332,2334,2336,2338,2340,2342,2344,2346,2348,2350,2352,2354,2356,2358,2360,2362,2364,2366,2368,2370,2372,2374,2376,2378,2380,2382,2384,2386,2388,2390,2392,2394,2396,2398,2400,2402,2404,2406,2408,2410,2412,2414,2416,2418,2420,2422,2424,2426,2428,2430,2432,2434,2436,2438,2440,2442,2444,2446,2448,2450,2452,2454,2456,2458,2460,2462,2464,2466,2468,2470,2472,2474,2476,2478,2480,2482,2484,2486,2488,2490,2492,2494,2496,2498,2500,2502,2504,2506,2508,2510,2512,2514,2516,2518,2520,2522,2524,2526,2528,2530,2532,2534,2536,2538,2540,2542,2544,2546,2548,2550,2552,2554,2556,2558,2560,2562,2564,2566,2568,2570,2572,2574,2576,2578,2580,2582,2584,2586,2588,2590,2592,2594,2596,2598,2600,2602,2604,2606,2608,2610,2612,2614,2616,2618,2620,2622,2624,2626,2628,2630,2632,2634,2636,2638,2640,2642,2644,2646,2648,2650,2652,2654,2656,2658,2660,2662,2664,2666,2668,2670,2672,2674,2676,2678,2680,2682,2684,2686,2688,2690,2692,2694,2696,2698,2700,2702,2704,2706,2708,2710,2712,2714,2716,2718,2720,2722,2724,2726,2728,2730,2732,2734,2736,2738,2740,2742,2744,2746,2748,2750,2752,2754,2756,2758,2760,2762,2764,2766,2768,2770,2772,2774,2776,2778,2780,2782,2784,2786,2788,2790,2792,2794,2796,2798,2800,2802,2804,2806,2808,2810,2812,2814,2816,2818,2820,2822,2824,2826,2828,2830,2832,2834,2836,2838,2840,2842,2844,2846,2848,2850,2852,2854,2856,2858,2860,2862,2864,2866,2868,2870,2872,2874,2876,2878,2880,2882,2884,2886,2888,2890,2892,2894,2896,2898,2900,2902,2904,2906,2908,2910,2912,2914,2916,2918,2920,2922,2924,2926,2928,2930,2932,2934,2936,2938,2940,2942,2944,2946,2948,2950,2952,2954,2956,2958,2960,2962,2964,2966,2968,2970,2972,2974,2976,2978,2980,2982,2984,2986,2988,2990,2992,2994,2996,2998,3000,3002,3004,3006,3008,3010,3012,3014,3016,3018,3020,3022,3024,3026,3028,3030,3032,3034,3036,3038,3040,3042,3044,3046,3048,3050,3052,3054,3056,3058,3060,3062,3064,3066,3068,3070,3072,3074,3076,3078,3080,3082,3084,3086,3088,3090,3092,3094,3096,3098,3100,3102,3104,3106,3108,3110,3112,3114,3116,3118,3120,3122,3124,3126,3128,3130,3132,3134,3136,3138,3140,3142,3144,3146,3148,3150,3152,3154,3156,3158,3160,3162,3164,3166,3168,3170,3172,3174,3176,3178,3180,3182,3184,3186,3188,3190,3192,3194,3196,3198,3200,3202,3204,3206,3208,3210,3212,3214,3216,3218,3220,3222,3224,3226,3228,3230,3232,3234,3236,3238,3240,3242,3244,3246,3248,3250,3252,3254,3256,3258,3260,3262,3264,3266,3268,3270,3272,3274,3276,3278,3280,3282,3284,3286,3288,3290,3292,3294,3296,3298,3300,3302,3304,3306,3308,3310,3312,3314,3316,3318,3320,3322,3324,3326,3328,3330,3332,3334,3336,3338,3340,3342,3344,3346,3348,3350,3352,3354,3356,3358,3360,3362,3364,3366,3368,3370,3372,3374,3376,3378,3380,3382,3384,3386,3388,3390,3392,3394,3396,3398,3400,3402,3404,3406,3408,3410,3412,3414,3416,3418,3420,3422,3424,3426,3428,3430,3432,3434,3436,3438,3440,3442,3444,3446,3448,3450,3452,3454,3456,3458,3460,3462,3464,3466,3468,3470,3472,3474,3476,3478,3480,3482,3484,3486,3488,3490,3492,3494,3496,3498,3500,3502,3504,3506,3508,3510,3512,3514,3516,3518,3520,3522,3524,3526,3528,3530,3532,3534,3536,3538,3540,3542,3544,3546,3548,3550,3552,3554,3556,3558,3560,3562,3564,3566,3568,3570,3572,3574,3576,3578,3580,3582,3584,3586,3588,3590,3592,3594,3596,3598,3600,3602,3604,3606,3608,3610,3612,3614,3616,3618,3620,3622,3624,3626,3628,3630,3632,3634,3636,3638,3640,3642,3644,3646,3648,3650,3652,3654,3656,3658,3660,3662,3664,3666,3668,3670,3672,3674,3676,3678],{"categories":97},[98],"Developer Productivity",{"categories":100},[101],"Business & SaaS",{"categories":103},[104],"AI & LLMs",{"categories":106},[107],"AI Automation",{"categories":109},[110],"Product Strategy",{"categories":112},[104],{"categories":114},[98],{"categories":116},[101],{"categories":118},[],{"categories":120},[104],{"categories":122},[],{"categories":124},[125],"AI News & Trends",{"categories":127},[107],{"categories":129},[125],{"categories":131},[107],{"categories":133},[107],{"categories":135},[104],{"categories":137},[104],{"categories":139},[125],{"categories":141},[104],{"categories":143},[],{"categories":145},[146],"Design & Frontend",{"categories":148},[149],"Data Science & Visualization",{"categories":151},[125],{"categories":153},[],{"categories":155},[156],"Software Engineering",{"categories":158},[104],{"categories":160},[107],{"categories":162},[163],"Marketing & Growth",{"categories":165},[104],{"categories":167},[107],{"categories":169},[],{"categories":171},[],{"categories":173},[146],{"categories":175},[107],{"categories":177},[98],{"categories":179},[146],{"categories":181},[104],{"categories":183},[107],{"categories":185},[125],{"categories":187},[],{"categories":189},[],{"categories":191},[107],{"categories":193},[156],{"categories":195},[],{"categories":197},[101],{"categories":199},[],{"categories":201},[],{"categories":203},[107],{"categories":205},[107],{"categories":207},[104],{"categories":209},[],{"categories":211},[156],{"categories":213},[],{"categories":215},[],{"categories":217},[],{"categories":219},[104],{"categories":221},[163],{"categories":223},[146],{"categories":225},[146],{"categories":227},[104],{"categories":229},[107],{"categories":231},[104],{"categories":233},[104],{"categories":235},[107],{"categories":237},[107],{"categories":239},[149],{"categories":241},[125],{"categories":243},[107],{"categories":245},[163],{"categories":247},[107],{"categories":249},[110],{"categories":251},[],{"categories":253},[107],{"categories":255},[],{"categories":257},[107],{"categories":259},[156],{"categories":261},[146],{"categories":263},[104],{"categories":265},[],{"categories":267},[],{"categories":269},[107],{"categories":271},[],{"categories":273},[104],{"categories":275},[],{"categories":277},[98],{"categories":279},[156],{"categories":281},[101],{"categories":283},[125],{"categories":285},[104],{"categories":287},[],{"categories":289},[104],{"categories":291},[],{"categories":293},[156],{"categories":295},[149],{"categories":297},[],{"categories":299},[104],{"categories":301},[146],{"categories":303},[],{"categories":305},[146],{"categories":307},[107],{"categories":309},[],{"categories":311},[107],{"categories":313},[125],{"categories":315},[101],{"categories":317},[104],{"categories":319},[],{"categories":321},[107],{"categories":323},[104],{"categories":325},[110],{"categories":327},[],{"categories":329},[104],{"categories":331},[107],{"categories":333},[107],{"categories":335},[],{"categories":337},[149],{"categories":339},[104],{"categories":341},[],{"categories":343},[98],{"categories":345},[101],{"categories":347},[104],{"categories":349},[107],{"categories":351},[156],{"categories":353},[104],{"categories":355},[],{"categories":357},[],{"categories":359},[104],{"categories":361},[],{"categories":363},[146],{"categories":365},[],{"categories":367},[104],{"categories":369},[],{"categories":371},[107],{"categories":373},[104],{"categories":375},[146],{"categories":377},[],{"categories":379},[104],{"categories":381},[104],{"categories":383},[101],{"categories":385},[107],{"categories":387},[104],{"categories":389},[146],{"categories":391},[107],{"categories":393},[],{"categories":395},[],{"categories":397},[125],{"categories":399},[],{"categories":401},[104],{"categories":403},[101,163],{"categories":405},[],{"categories":407},[104],{"categories":409},[],{"categories":411},[],{"categories":413},[104],{"categories":415},[],{"categories":417},[104],{"categories":419},[56],{"categories":421},[],{"categories":423},[125],{"categories":425},[146],{"categories":427},[],{"categories":429},[125],{"categories":431},[125],{"categories":433},[104],{"categories":435},[163],{"categories":437},[],{"categories":439},[101],{"categories":441},[],{"categories":443},[104,56],{"categories":445},[104],{"categories":447},[104],{"categories":449},[107],{"categories":451},[104,156],{"categories":453},[149],{"categories":455},[104],{"categories":457},[163],{"categories":459},[107],{"categories":461},[107],{"categories":463},[],{"categories":465},[107],{"categories":467},[104,101],{"categories":469},[],{"categories":471},[146],{"categories":473},[146],{"categories":475},[],{"categories":477},[],{"categories":479},[125],{"categories":481},[],{"categories":483},[98],{"categories":485},[156],{"categories":487},[104],{"categories":489},[146],{"categories":491},[107],{"categories":493},[156],{"categories":495},[125],{"categories":497},[146],{"categories":499},[],{"categories":501},[104],{"categories":503},[104],{"categories":505},[104],{"categories":507},[125],{"categories":509},[98],{"categories":511},[104],{"categories":513},[107],{"categories":515},[56],{"categories":517},[146],{"categories":519},[107],{"categories":521},[],{"categories":523},[],{"categories":525},[146],{"categories":527},[125],{"categories":529},[149],{"categories":531},[],{"categories":533},[104],{"categories":535},[104],{"categories":537},[101],{"categories":539},[104],{"categories":541},[104],{"categories":543},[125],{"categories":545},[],{"categories":547},[107],{"categories":549},[156],{"categories":551},[],{"categories":553},[104],{"categories":555},[104],{"categories":557},[107],{"categories":559},[],{"categories":561},[],{"categories":563},[104],{"categories":565},[],{"categories":567},[101],{"categories":569},[107],{"categories":571},[],{"categories":573},[98],{"categories":575},[104],{"categories":577},[101],{"categories":579},[125],{"categories":581},[],{"categories":583},[],{"categories":585},[],{"categories":587},[125],{"categories":589},[125],{"categories":591},[],{"categories":593},[],{"categories":595},[101],{"categories":597},[],{"categories":599},[],{"categories":601},[98],{"categories":603},[],{"categories":605},[163],{"categories":607},[107],{"categories":609},[101],{"categories":611},[107],{"categories":613},[156],{"categories":615},[],{"categories":617},[110],{"categories":619},[146],{"categories":621},[156],{"categories":623},[104],{"categories":625},[107],{"categories":627},[101],{"categories":629},[104],{"categories":631},[],{"categories":633},[],{"categories":635},[156],{"categories":637},[149],{"categories":639},[110],{"categories":641},[107],{"categories":643},[104],{"categories":645},[],{"categories":647},[56],{"categories":649},[],{"categories":651},[107],{"categories":653},[],{"categories":655},[],{"categories":657},[104],{"categories":659},[146],{"categories":661},[163],{"categories":663},[107],{"categories":665},[],{"categories":667},[98],{"categories":669},[],{"categories":671},[125],{"categories":673},[104,56],{"categories":675},[125],{"categories":677},[104],{"categories":679},[101],{"categories":681},[104],{"categories":683},[],{"categories":685},[101],{"categories":687},[],{"categories":689},[156],{"categories":691},[146],{"categories":693},[125],{"categories":695},[149],{"categories":697},[98],{"categories":699},[104],{"categories":701},[156],{"categories":703},[],{"categories":705},[],{"categories":707},[110],{"categories":709},[],{"categories":711},[104],{"categories":713},[],{"categories":715},[146],{"categories":717},[146],{"categories":719},[146],{"categories":721},[],{"categories":723},[],{"categories":725},[125],{"categories":727},[107],{"categories":729},[104],{"categories":731},[104],{"categories":733},[104],{"categories":735},[101],{"categories":737},[104],{"categories":739},[],{"categories":741},[156],{"categories":743},[156],{"categories":745},[101],{"categories":747},[],{"categories":749},[104],{"categories":751},[104],{"categories":753},[101],{"categories":755},[125],{"categories":757},[163],{"categories":759},[107],{"categories":761},[],{"categories":763},[146],{"categories":765},[],{"categories":767},[104],{"categories":769},[],{"categories":771},[101],{"categories":773},[107],{"categories":775},[],{"categories":777},[56],{"categories":779},[149],{"categories":781},[156],{"categories":783},[163],{"categories":785},[156],{"categories":787},[107],{"categories":789},[],{"categories":791},[],{"categories":793},[107],{"categories":795},[98],{"categories":797},[107],{"categories":799},[110],{"categories":801},[101],{"categories":803},[],{"categories":805},[104],{"categories":807},[110],{"categories":809},[104],{"categories":811},[104],{"categories":813},[163],{"categories":815},[146],{"categories":817},[107],{"categories":819},[],{"categories":821},[],{"categories":823},[56],{"categories":825},[156],{"categories":827},[],{"categories":829},[107],{"categories":831},[104],{"categories":833},[146,104],{"categories":835},[98],{"categories":837},[],{"categories":839},[104],{"categories":841},[98],{"categories":843},[146],{"categories":845},[107],{"categories":847},[156],{"categories":849},[],{"categories":851},[104],{"categories":853},[],{"categories":855},[98],{"categories":857},[],{"categories":859},[107],{"categories":861},[110],{"categories":863},[104],{"categories":865},[104],{"categories":867},[146],{"categories":869},[107],{"categories":871},[56],{"categories":873},[146],{"categories":875},[107],{"categories":877},[104],{"categories":879},[104],{"categories":881},[104],{"categories":883},[125],{"categories":885},[],{"categories":887},[110],{"categories":889},[107],{"categories":891},[146],{"categories":893},[107],{"categories":895},[156],{"categories":897},[146],{"categories":899},[107],{"categories":901},[125],{"categories":903},[],{"categories":905},[104],{"categories":907},[146],{"categories":909},[104],{"categories":911},[98],{"categories":913},[125],{"categories":915},[104],{"categories":917},[163],{"categories":919},[104],{"categories":921},[104],{"categories":923},[107],{"categories":925},[107],{"categories":927},[104],{"categories":929},[107],{"categories":931},[146],{"categories":933},[104],{"categories":935},[],{"categories":937},[],{"categories":939},[156],{"categories":941},[],{"categories":943},[98],{"categories":945},[56],{"categories":947},[],{"categories":949},[98],{"categories":951},[101],{"categories":953},[163],{"categories":955},[],{"categories":957},[101],{"categories":959},[],{"categories":961},[],{"categories":963},[],{"categories":965},[],{"categories":967},[],{"categories":969},[104],{"categories":971},[107],{"categories":973},[56],{"categories":975},[98],{"categories":977},[104],{"categories":979},[156],{"categories":981},[110],{"categories":983},[104],{"categories":985},[163],{"categories":987},[104],{"categories":989},[104],{"categories":991},[104],{"categories":993},[104,98],{"categories":995},[156],{"categories":997},[156],{"categories":999},[146],{"categories":1001},[104],{"categories":1003},[],{"categories":1005},[],{"categories":1007},[],{"categories":1009},[156],{"categories":1011},[149],{"categories":1013},[125],{"categories":1015},[146],{"categories":1017},[],{"categories":1019},[104],{"categories":1021},[104],{"categories":1023},[],{"categories":1025},[],{"categories":1027},[107],{"categories":1029},[104],{"categories":1031},[101],{"categories":1033},[],{"categories":1035},[98],{"categories":1037},[104],{"categories":1039},[98],{"categories":1041},[104],{"categories":1043},[156],{"categories":1045},[163],{"categories":1047},[104,146],{"categories":1049},[125],{"categories":1051},[146],{"categories":1053},[],{"categories":1055},[56],{"categories":1057},[146],{"categories":1059},[107],{"categories":1061},[],{"categories":1063},[],{"categories":1065},[],{"categories":1067},[],{"categories":1069},[156],{"categories":1071},[107],{"categories":1073},[107],{"categories":1075},[56],{"categories":1077},[104],{"categories":1079},[104],{"categories":1081},[104],{"categories":1083},[],{"categories":1085},[146],{"categories":1087},[],{"categories":1089},[],{"categories":1091},[107],{"categories":1093},[],{"categories":1095},[],{"categories":1097},[163],{"categories":1099},[163],{"categories":1101},[107],{"categories":1103},[],{"categories":1105},[104],{"categories":1107},[104],{"categories":1109},[156],{"categories":1111},[146],{"categories":1113},[146],{"categories":1115},[107],{"categories":1117},[98],{"categories":1119},[104],{"categories":1121},[146],{"categories":1123},[146],{"categories":1125},[107],{"categories":1127},[107],{"categories":1129},[104],{"categories":1131},[],{"categories":1133},[],{"categories":1135},[104],{"categories":1137},[107],{"categories":1139},[125],{"categories":1141},[156],{"categories":1143},[98],{"categories":1145},[104],{"categories":1147},[],{"categories":1149},[107],{"categories":1151},[107],{"categories":1153},[],{"categories":1155},[98],{"categories":1157},[104],{"categories":1159},[98],{"categories":1161},[98],{"categories":1163},[],{"categories":1165},[],{"categories":1167},[107],{"categories":1169},[107],{"categories":1171},[104],{"categories":1173},[104],{"categories":1175},[125],{"categories":1177},[149],{"categories":1179},[110],{"categories":1181},[125],{"categories":1183},[146],{"categories":1185},[],{"categories":1187},[125],{"categories":1189},[],{"categories":1191},[],{"categories":1193},[],{"categories":1195},[],{"categories":1197},[156],{"categories":1199},[149],{"categories":1201},[],{"categories":1203},[104],{"categories":1205},[104],{"categories":1207},[149],{"categories":1209},[156],{"categories":1211},[],{"categories":1213},[],{"categories":1215},[107],{"categories":1217},[125],{"categories":1219},[125],{"categories":1221},[107],{"categories":1223},[98],{"categories":1225},[104,56],{"categories":1227},[],{"categories":1229},[146],{"categories":1231},[98],{"categories":1233},[107],{"categories":1235},[146],{"categories":1237},[],{"categories":1239},[107],{"categories":1241},[107],{"categories":1243},[104],{"categories":1245},[163],{"categories":1247},[156],{"categories":1249},[146],{"categories":1251},[],{"categories":1253},[107],{"categories":1255},[104],{"categories":1257},[107],{"categories":1259},[107],{"categories":1261},[107],{"categories":1263},[163],{"categories":1265},[107],{"categories":1267},[104],{"categories":1269},[],{"categories":1271},[163],{"categories":1273},[125],{"categories":1275},[107],{"categories":1277},[],{"categories":1279},[],{"categories":1281},[104],{"categories":1283},[107],{"categories":1285},[125],{"categories":1287},[107],{"categories":1289},[],{"categories":1291},[],{"categories":1293},[],{"categories":1295},[107],{"categories":1297},[],{"categories":1299},[],{"categories":1301},[149],{"categories":1303},[104],{"categories":1305},[149],{"categories":1307},[125],{"categories":1309},[104],{"categories":1311},[104],{"categories":1313},[107],{"categories":1315},[104],{"categories":1317},[],{"categories":1319},[],{"categories":1321},[56],{"categories":1323},[],{"categories":1325},[],{"categories":1327},[98],{"categories":1329},[],{"categories":1331},[],{"categories":1333},[],{"categories":1335},[],{"categories":1337},[156],{"categories":1339},[125],{"categories":1341},[163],{"categories":1343},[101],{"categories":1345},[104],{"categories":1347},[104],{"categories":1349},[101],{"categories":1351},[],{"categories":1353},[146],{"categories":1355},[107],{"categories":1357},[101],{"categories":1359},[104],{"categories":1361},[104],{"categories":1363},[98],{"categories":1365},[],{"categories":1367},[98],{"categories":1369},[104],{"categories":1371},[163],{"categories":1373},[107],{"categories":1375},[125],{"categories":1377},[101],{"categories":1379},[104],{"categories":1381},[107],{"categories":1383},[],{"categories":1385},[104],{"categories":1387},[98],{"categories":1389},[104],{"categories":1391},[],{"categories":1393},[125],{"categories":1395},[104],{"categories":1397},[],{"categories":1399},[101],{"categories":1401},[104],{"categories":1403},[],{"categories":1405},[],{"categories":1407},[],{"categories":1409},[104],{"categories":1411},[],{"categories":1413},[56],{"categories":1415},[104],{"categories":1417},[],{"categories":1419},[104],{"categories":1421},[104],{"categories":1423},[104],{"categories":1425},[104,56],{"categories":1427},[104],{"categories":1429},[104],{"categories":1431},[146],{"categories":1433},[107],{"categories":1435},[],{"categories":1437},[107],{"categories":1439},[104],{"categories":1441},[104],{"categories":1443},[104],{"categories":1445},[98],{"categories":1447},[98],{"categories":1449},[156],{"categories":1451},[146],{"categories":1453},[107],{"categories":1455},[],{"categories":1457},[104],{"categories":1459},[125],{"categories":1461},[104],{"categories":1463},[101],{"categories":1465},[],{"categories":1467},[56],{"categories":1469},[146],{"categories":1471},[146],{"categories":1473},[107],{"categories":1475},[125],{"categories":1477},[107],{"categories":1479},[104],{"categories":1481},[],{"categories":1483},[104],{"categories":1485},[],{"categories":1487},[],{"categories":1489},[104],{"categories":1491},[104],{"categories":1493},[104],{"categories":1495},[107],{"categories":1497},[104],{"categories":1499},[],{"categories":1501},[149],{"categories":1503},[107],{"categories":1505},[],{"categories":1507},[],{"categories":1509},[104],{"categories":1511},[125],{"categories":1513},[],{"categories":1515},[146],{"categories":1517},[56],{"categories":1519},[125],{"categories":1521},[156],{"categories":1523},[156],{"categories":1525},[125],{"categories":1527},[125],{"categories":1529},[56],{"categories":1531},[],{"categories":1533},[125],{"categories":1535},[104],{"categories":1537},[98],{"categories":1539},[125],{"categories":1541},[],{"categories":1543},[149],{"categories":1545},[125],{"categories":1547},[156],{"categories":1549},[125],{"categories":1551},[56],{"categories":1553},[104],{"categories":1555},[104],{"categories":1557},[],{"categories":1559},[101],{"categories":1561},[],{"categories":1563},[],{"categories":1565},[104],{"categories":1567},[104],{"categories":1569},[104],{"categories":1571},[104],{"categories":1573},[],{"categories":1575},[149],{"categories":1577},[98],{"categories":1579},[],{"categories":1581},[104],{"categories":1583},[104],{"categories":1585},[56],{"categories":1587},[56],{"categories":1589},[],{"categories":1591},[107],{"categories":1593},[125],{"categories":1595},[125],{"categories":1597},[104],{"categories":1599},[107],{"categories":1601},[],{"categories":1603},[146],{"categories":1605},[104],{"categories":1607},[104],{"categories":1609},[],{"categories":1611},[],{"categories":1613},[56],{"categories":1615},[104],{"categories":1617},[156],{"categories":1619},[101],{"categories":1621},[104],{"categories":1623},[],{"categories":1625},[107],{"categories":1627},[98],{"categories":1629},[98],{"categories":1631},[],{"categories":1633},[104],{"categories":1635},[146],{"categories":1637},[107],{"categories":1639},[],{"categories":1641},[104],{"categories":1643},[104],{"categories":1645},[107],{"categories":1647},[],{"categories":1649},[107],{"categories":1651},[156],{"categories":1653},[],{"categories":1655},[104],{"categories":1657},[],{"categories":1659},[104],{"categories":1661},[],{"categories":1663},[104],{"categories":1665},[104],{"categories":1667},[],{"categories":1669},[104],{"categories":1671},[125],{"categories":1673},[104],{"categories":1675},[104],{"categories":1677},[98],{"categories":1679},[104],{"categories":1681},[125],{"categories":1683},[107],{"categories":1685},[],{"categories":1687},[104],{"categories":1689},[163],{"categories":1691},[],{"categories":1693},[],{"categories":1695},[],{"categories":1697},[98],{"categories":1699},[125],{"categories":1701},[107],{"categories":1703},[104],{"categories":1705},[146],{"categories":1707},[107],{"categories":1709},[],{"categories":1711},[107],{"categories":1713},[],{"categories":1715},[104],{"categories":1717},[107],{"categories":1719},[104],{"categories":1721},[],{"categories":1723},[104],{"categories":1725},[104],{"categories":1727},[125],{"categories":1729},[146],{"categories":1731},[107],{"categories":1733},[146],{"categories":1735},[101],{"categories":1737},[],{"categories":1739},[],{"categories":1741},[104],{"categories":1743},[98],{"categories":1745},[125],{"categories":1747},[],{"categories":1749},[],{"categories":1751},[156],{"categories":1753},[146],{"categories":1755},[],{"categories":1757},[104],{"categories":1759},[],{"categories":1761},[163],{"categories":1763},[104],{"categories":1765},[56],{"categories":1767},[156],{"categories":1769},[],{"categories":1771},[107],{"categories":1773},[104],{"categories":1775},[107],{"categories":1777},[107],{"categories":1779},[104],{"categories":1781},[],{"categories":1783},[98],{"categories":1785},[104],{"categories":1787},[101],{"categories":1789},[156],{"categories":1791},[146],{"categories":1793},[],{"categories":1795},[],{"categories":1797},[],{"categories":1799},[107],{"categories":1801},[146],{"categories":1803},[125],{"categories":1805},[104],{"categories":1807},[125],{"categories":1809},[146],{"categories":1811},[],{"categories":1813},[146],{"categories":1815},[125],{"categories":1817},[101],{"categories":1819},[104],{"categories":1821},[125],{"categories":1823},[163],{"categories":1825},[],{"categories":1827},[],{"categories":1829},[149],{"categories":1831},[104,156],{"categories":1833},[125],{"categories":1835},[104],{"categories":1837},[107],{"categories":1839},[107],{"categories":1841},[104],{"categories":1843},[],{"categories":1845},[156],{"categories":1847},[104],{"categories":1849},[149],{"categories":1851},[107],{"categories":1853},[163],{"categories":1855},[56],{"categories":1857},[],{"categories":1859},[98],{"categories":1861},[107],{"categories":1863},[107],{"categories":1865},[156],{"categories":1867},[104],{"categories":1869},[104],{"categories":1871},[],{"categories":1873},[],{"categories":1875},[],{"categories":1877},[56],{"categories":1879},[125],{"categories":1881},[104],{"categories":1883},[104],{"categories":1885},[104],{"categories":1887},[],{"categories":1889},[149],{"categories":1891},[101],{"categories":1893},[],{"categories":1895},[107],{"categories":1897},[56],{"categories":1899},[],{"categories":1901},[146],{"categories":1903},[146],{"categories":1905},[],{"categories":1907},[156],{"categories":1909},[146],{"categories":1911},[104],{"categories":1913},[],{"categories":1915},[125],{"categories":1917},[104],{"categories":1919},[146],{"categories":1921},[107],{"categories":1923},[125],{"categories":1925},[],{"categories":1927},[107],{"categories":1929},[146],{"categories":1931},[104],{"categories":1933},[],{"categories":1935},[104],{"categories":1937},[104],{"categories":1939},[56],{"categories":1941},[125],{"categories":1943},[149],{"categories":1945},[149],{"categories":1947},[],{"categories":1949},[],{"categories":1951},[],{"categories":1953},[107],{"categories":1955},[156],{"categories":1957},[156],{"categories":1959},[],{"categories":1961},[],{"categories":1963},[104],{"categories":1965},[],{"categories":1967},[107],{"categories":1969},[104],{"categories":1971},[],{"categories":1973},[104],{"categories":1975},[101],{"categories":1977},[104],{"categories":1979},[163],{"categories":1981},[107],{"categories":1983},[104],{"categories":1985},[156],{"categories":1987},[125],{"categories":1989},[107],{"categories":1991},[],{"categories":1993},[125],{"categories":1995},[107],{"categories":1997},[107],{"categories":1999},[],{"categories":2001},[101],{"categories":2003},[107],{"categories":2005},[],{"categories":2007},[104],{"categories":2009},[98],{"categories":2011},[125],{"categories":2013},[56],{"categories":2015},[107],{"categories":2017},[107],{"categories":2019},[98],{"categories":2021},[104],{"categories":2023},[],{"categories":2025},[],{"categories":2027},[146],{"categories":2029},[104,101],{"categories":2031},[],{"categories":2033},[98],{"categories":2035},[149],{"categories":2037},[104],{"categories":2039},[156],{"categories":2041},[104],{"categories":2043},[107],{"categories":2045},[104],{"categories":2047},[104],{"categories":2049},[125],{"categories":2051},[107],{"categories":2053},[],{"categories":2055},[],{"categories":2057},[107],{"categories":2059},[104],{"categories":2061},[56],{"categories":2063},[],{"categories":2065},[104],{"categories":2067},[107],{"categories":2069},[],{"categories":2071},[104],{"categories":2073},[163],{"categories":2075},[149],{"categories":2077},[107],{"categories":2079},[104],{"categories":2081},[56],{"categories":2083},[],{"categories":2085},[104],{"categories":2087},[163],{"categories":2089},[146],{"categories":2091},[104],{"categories":2093},[],{"categories":2095},[163],{"categories":2097},[125],{"categories":2099},[104],{"categories":2101},[104],{"categories":2103},[98],{"categories":2105},[],{"categories":2107},[],{"categories":2109},[146],{"categories":2111},[104],{"categories":2113},[149],{"categories":2115},[163],{"categories":2117},[163],{"categories":2119},[125],{"categories":2121},[],{"categories":2123},[],{"categories":2125},[104],{"categories":2127},[],{"categories":2129},[104,156],{"categories":2131},[125],{"categories":2133},[107],{"categories":2135},[156],{"categories":2137},[104],{"categories":2139},[98],{"categories":2141},[],{"categories":2143},[],{"categories":2145},[98],{"categories":2147},[163],{"categories":2149},[104],{"categories":2151},[],{"categories":2153},[146,104],{"categories":2155},[56],{"categories":2157},[98],{"categories":2159},[],{"categories":2161},[101],{"categories":2163},[101],{"categories":2165},[104],{"categories":2167},[156],{"categories":2169},[107],{"categories":2171},[125],{"categories":2173},[163],{"categories":2175},[146],{"categories":2177},[104],{"categories":2179},[104],{"categories":2181},[104],{"categories":2183},[98],{"categories":2185},[104],{"categories":2187},[107],{"categories":2189},[125],{"categories":2191},[],{"categories":2193},[],{"categories":2195},[149],{"categories":2197},[156],{"categories":2199},[104],{"categories":2201},[146],{"categories":2203},[149],{"categories":2205},[104],{"categories":2207},[104],{"categories":2209},[107],{"categories":2211},[107],{"categories":2213},[104,101],{"categories":2215},[],{"categories":2217},[146],{"categories":2219},[],{"categories":2221},[104],{"categories":2223},[125],{"categories":2225},[98],{"categories":2227},[98],{"categories":2229},[107],{"categories":2231},[104],{"categories":2233},[101],{"categories":2235},[156],{"categories":2237},[163],{"categories":2239},[],{"categories":2241},[125],{"categories":2243},[104],{"categories":2245},[104],{"categories":2247},[125],{"categories":2249},[156],{"categories":2251},[104],{"categories":2253},[107],{"categories":2255},[125],{"categories":2257},[104],{"categories":2259},[146],{"categories":2261},[104],{"categories":2263},[104],{"categories":2265},[56],{"categories":2267},[110],{"categories":2269},[107],{"categories":2271},[104],{"categories":2273},[125],{"categories":2275},[107],{"categories":2277},[163],{"categories":2279},[104],{"categories":2281},[],{"categories":2283},[104],{"categories":2285},[],{"categories":2287},[],{"categories":2289},[],{"categories":2291},[101],{"categories":2293},[104],{"categories":2295},[107],{"categories":2297},[125],{"categories":2299},[125],{"categories":2301},[125],{"categories":2303},[125],{"categories":2305},[],{"categories":2307},[98],{"categories":2309},[107],{"categories":2311},[125],{"categories":2313},[98],{"categories":2315},[107],{"categories":2317},[104],{"categories":2319},[104,107],{"categories":2321},[107],{"categories":2323},[56],{"categories":2325},[125],{"categories":2327},[125],{"categories":2329},[107],{"categories":2331},[104],{"categories":2333},[],{"categories":2335},[125],{"categories":2337},[163],{"categories":2339},[98],{"categories":2341},[104],{"categories":2343},[104],{"categories":2345},[],{"categories":2347},[156],{"categories":2349},[],{"categories":2351},[98],{"categories":2353},[107],{"categories":2355},[125],{"categories":2357},[104],{"categories":2359},[125],{"categories":2361},[98],{"categories":2363},[125],{"categories":2365},[125],{"categories":2367},[],{"categories":2369},[101],{"categories":2371},[107],{"categories":2373},[125],{"categories":2375},[125],{"categories":2377},[125],{"categories":2379},[125],{"categories":2381},[125],{"categories":2383},[125],{"categories":2385},[125],{"categories":2387},[125],{"categories":2389},[125],{"categories":2391},[125],{"categories":2393},[149],{"categories":2395},[98],{"categories":2397},[104],{"categories":2399},[104],{"categories":2401},[],{"categories":2403},[104,98],{"categories":2405},[],{"categories":2407},[107],{"categories":2409},[125],{"categories":2411},[107],{"categories":2413},[104],{"categories":2415},[104],{"categories":2417},[104],{"categories":2419},[104],{"categories":2421},[104],{"categories":2423},[107],{"categories":2425},[101],{"categories":2427},[146],{"categories":2429},[125],{"categories":2431},[104],{"categories":2433},[],{"categories":2435},[],{"categories":2437},[107],{"categories":2439},[146],{"categories":2441},[104],{"categories":2443},[],{"categories":2445},[],{"categories":2447},[163],{"categories":2449},[104],{"categories":2451},[],{"categories":2453},[],{"categories":2455},[98],{"categories":2457},[101],{"categories":2459},[104],{"categories":2461},[101],{"categories":2463},[146],{"categories":2465},[],{"categories":2467},[125],{"categories":2469},[],{"categories":2471},[146],{"categories":2473},[104],{"categories":2475},[163],{"categories":2477},[],{"categories":2479},[163],{"categories":2481},[],{"categories":2483},[],{"categories":2485},[107],{"categories":2487},[],{"categories":2489},[101],{"categories":2491},[98],{"categories":2493},[146],{"categories":2495},[156],{"categories":2497},[],{"categories":2499},[],{"categories":2501},[104],{"categories":2503},[98],{"categories":2505},[163],{"categories":2507},[],{"categories":2509},[107],{"categories":2511},[107],{"categories":2513},[125],{"categories":2515},[104],{"categories":2517},[107],{"categories":2519},[104],{"categories":2521},[107],{"categories":2523},[104],{"categories":2525},[110],{"categories":2527},[125],{"categories":2529},[],{"categories":2531},[163],{"categories":2533},[156],{"categories":2535},[107],{"categories":2537},[],{"categories":2539},[104],{"categories":2541},[107],{"categories":2543},[101],{"categories":2545},[98],{"categories":2547},[104],{"categories":2549},[146],{"categories":2551},[156],{"categories":2553},[156],{"categories":2555},[104],{"categories":2557},[149],{"categories":2559},[104],{"categories":2561},[107],{"categories":2563},[101],{"categories":2565},[107],{"categories":2567},[104],{"categories":2569},[104],{"categories":2571},[107],{"categories":2573},[125],{"categories":2575},[],{"categories":2577},[98],{"categories":2579},[104],{"categories":2581},[107],{"categories":2583},[104],{"categories":2585},[104],{"categories":2587},[],{"categories":2589},[146],{"categories":2591},[101],{"categories":2593},[125],{"categories":2595},[104],{"categories":2597},[104],{"categories":2599},[146],{"categories":2601},[163],{"categories":2603},[149],{"categories":2605},[104],{"categories":2607},[125],{"categories":2609},[104],{"categories":2611},[107],{"categories":2613},[56],{"categories":2615},[104],{"categories":2617},[107],{"categories":2619},[149],{"categories":2621},[],{"categories":2623},[107],{"categories":2625},[156],{"categories":2627},[146],{"categories":2629},[104],{"categories":2631},[98],{"categories":2633},[101],{"categories":2635},[156],{"categories":2637},[],{"categories":2639},[107],{"categories":2641},[104],{"categories":2643},[],{"categories":2645},[125],{"categories":2647},[],{"categories":2649},[125],{"categories":2651},[104],{"categories":2653},[107],{"categories":2655},[107],{"categories":2657},[107],{"categories":2659},[],{"categories":2661},[],{"categories":2663},[104],{"categories":2665},[104],{"categories":2667},[],{"categories":2669},[146],{"categories":2671},[107],{"categories":2673},[163],{"categories":2675},[98],{"categories":2677},[],{"categories":2679},[],{"categories":2681},[125],{"categories":2683},[156],{"categories":2685},[104],{"categories":2687},[104],{"categories":2689},[104],{"categories":2691},[156],{"categories":2693},[125],{"categories":2695},[146],{"categories":2697},[104],{"categories":2699},[104],{"categories":2701},[104],{"categories":2703},[125],{"categories":2705},[104],{"categories":2707},[125],{"categories":2709},[107],{"categories":2711},[107],{"categories":2713},[156],{"categories":2715},[107],{"categories":2717},[104],{"categories":2719},[156],{"categories":2721},[146],{"categories":2723},[],{"categories":2725},[107],{"categories":2727},[],{"categories":2729},[],{"categories":2731},[],{"categories":2733},[101],{"categories":2735},[104],{"categories":2737},[107],{"categories":2739},[98],{"categories":2741},[107],{"categories":2743},[163],{"categories":2745},[],{"categories":2747},[107],{"categories":2749},[],{"categories":2751},[98],{"categories":2753},[107],{"categories":2755},[],{"categories":2757},[107],{"categories":2759},[104],{"categories":2761},[125],{"categories":2763},[104],{"categories":2765},[107],{"categories":2767},[125],{"categories":2769},[107],{"categories":2771},[156],{"categories":2773},[146],{"categories":2775},[98],{"categories":2777},[],{"categories":2779},[107],{"categories":2781},[146],{"categories":2783},[56],{"categories":2785},[125],{"categories":2787},[104],{"categories":2789},[146],{"categories":2791},[98],{"categories":2793},[],{"categories":2795},[107],{"categories":2797},[107],{"categories":2799},[104],{"categories":2801},[],{"categories":2803},[107],{"categories":2805},[110],{"categories":2807},[125],{"categories":2809},[107],{"categories":2811},[101],{"categories":2813},[],{"categories":2815},[104],{"categories":2817},[110],{"categories":2819},[104],{"categories":2821},[107],{"categories":2823},[125],{"categories":2825},[98],{"categories":2827},[56],{"categories":2829},[104],{"categories":2831},[104],{"categories":2833},[104],{"categories":2835},[125],{"categories":2837},[101],{"categories":2839},[104],{"categories":2841},[146],{"categories":2843},[125],{"categories":2845},[56],{"categories":2847},[104],{"categories":2849},[],{"categories":2851},[],{"categories":2853},[56],{"categories":2855},[149],{"categories":2857},[107],{"categories":2859},[107],{"categories":2861},[125],{"categories":2863},[104],{"categories":2865},[98],{"categories":2867},[146],{"categories":2869},[107],{"categories":2871},[104],{"categories":2873},[163],{"categories":2875},[104],{"categories":2877},[107],{"categories":2879},[],{"categories":2881},[104],{"categories":2883},[104],{"categories":2885},[125],{"categories":2887},[98],{"categories":2889},[],{"categories":2891},[104],{"categories":2893},[104],{"categories":2895},[156],{"categories":2897},[146],{"categories":2899},[104,107],{"categories":2901},[163,101],{"categories":2903},[104],{"categories":2905},[],{"categories":2907},[107],{"categories":2909},[],{"categories":2911},[156],{"categories":2913},[104],{"categories":2915},[125],{"categories":2917},[],{"categories":2919},[107],{"categories":2921},[],{"categories":2923},[146],{"categories":2925},[107],{"categories":2927},[98],{"categories":2929},[107],{"categories":2931},[104],{"categories":2933},[56],{"categories":2935},[163],{"categories":2937},[101],{"categories":2939},[101],{"categories":2941},[98],{"categories":2943},[98],{"categories":2945},[104],{"categories":2947},[107],{"categories":2949},[104],{"categories":2951},[104],{"categories":2953},[98],{"categories":2955},[104],{"categories":2957},[163],{"categories":2959},[125],{"categories":2961},[104],{"categories":2963},[107],{"categories":2965},[104],{"categories":2967},[],{"categories":2969},[156],{"categories":2971},[],{"categories":2973},[107],{"categories":2975},[98],{"categories":2977},[],{"categories":2979},[56],{"categories":2981},[104],{"categories":2983},[],{"categories":2985},[125],{"categories":2987},[107],{"categories":2989},[156],{"categories":2991},[104],{"categories":2993},[107],{"categories":2995},[156],{"categories":2997},[107],{"categories":2999},[125],{"categories":3001},[98],{"categories":3003},[125],{"categories":3005},[156],{"categories":3007},[104],{"categories":3009},[146],{"categories":3011},[104],{"categories":3013},[104],{"categories":3015},[104],{"categories":3017},[104],{"categories":3019},[107],{"categories":3021},[104],{"categories":3023},[107],{"categories":3025},[104],{"categories":3027},[98],{"categories":3029},[104],{"categories":3031},[107],{"categories":3033},[146],{"categories":3035},[98],{"categories":3037},[107],{"categories":3039},[146],{"categories":3041},[],{"categories":3043},[104],{"categories":3045},[104],{"categories":3047},[156],{"categories":3049},[],{"categories":3051},[107],{"categories":3053},[163],{"categories":3055},[104],{"categories":3057},[125],{"categories":3059},[163],{"categories":3061},[107],{"categories":3063},[101],{"categories":3065},[101],{"categories":3067},[104],{"categories":3069},[98],{"categories":3071},[],{"categories":3073},[104],{"categories":3075},[],{"categories":3077},[98],{"categories":3079},[104],{"categories":3081},[107],{"categories":3083},[107],{"categories":3085},[],{"categories":3087},[156],{"categories":3089},[156],{"categories":3091},[163],{"categories":3093},[146],{"categories":3095},[],{"categories":3097},[104],{"categories":3099},[98],{"categories":3101},[104],{"categories":3103},[156],{"categories":3105},[98],{"categories":3107},[125],{"categories":3109},[125],{"categories":3111},[],{"categories":3113},[125],{"categories":3115},[107],{"categories":3117},[146],{"categories":3119},[149],{"categories":3121},[104],{"categories":3123},[],{"categories":3125},[125],{"categories":3127},[156],{"categories":3129},[101],{"categories":3131},[104],{"categories":3133},[98],{"categories":3135},[56],{"categories":3137},[98],{"categories":3139},[],{"categories":3141},[],{"categories":3143},[125],{"categories":3145},[],{"categories":3147},[107],{"categories":3149},[107],{"categories":3151},[107],{"categories":3153},[],{"categories":3155},[104],{"categories":3157},[],{"categories":3159},[125],{"categories":3161},[98],{"categories":3163},[146],{"categories":3165},[104],{"categories":3167},[125],{"categories":3169},[125],{"categories":3171},[],{"categories":3173},[125],{"categories":3175},[98],{"categories":3177},[104],{"categories":3179},[],{"categories":3181},[107],{"categories":3183},[107],{"categories":3185},[98],{"categories":3187},[],{"categories":3189},[],{"categories":3191},[],{"categories":3193},[146],{"categories":3195},[107],{"categories":3197},[104],{"categories":3199},[],{"categories":3201},[],{"categories":3203},[],{"categories":3205},[146],{"categories":3207},[],{"categories":3209},[98],{"categories":3211},[],{"categories":3213},[],{"categories":3215},[146],{"categories":3217},[104],{"categories":3219},[125],{"categories":3221},[],{"categories":3223},[163],{"categories":3225},[125],{"categories":3227},[163],{"categories":3229},[104],{"categories":3231},[],{"categories":3233},[],{"categories":3235},[107],{"categories":3237},[],{"categories":3239},[],{"categories":3241},[107],{"categories":3243},[104],{"categories":3245},[],{"categories":3247},[107],{"categories":3249},[125],{"categories":3251},[163],{"categories":3253},[149],{"categories":3255},[107],{"categories":3257},[107],{"categories":3259},[],{"categories":3261},[],{"categories":3263},[],{"categories":3265},[125],{"categories":3267},[],{"categories":3269},[],{"categories":3271},[146],{"categories":3273},[98],{"categories":3275},[],{"categories":3277},[101],{"categories":3279},[163],{"categories":3281},[104],{"categories":3283},[156],{"categories":3285},[98],{"categories":3287},[149],{"categories":3289},[101],{"categories":3291},[156],{"categories":3293},[],{"categories":3295},[],{"categories":3297},[107],{"categories":3299},[98],{"categories":3301},[146],{"categories":3303},[98],{"categories":3305},[107],{"categories":3307},[56],{"categories":3309},[107],{"categories":3311},[],{"categories":3313},[104],{"categories":3315},[125],{"categories":3317},[156],{"categories":3319},[],{"categories":3321},[146],{"categories":3323},[125],{"categories":3325},[98],{"categories":3327},[107],{"categories":3329},[104],{"categories":3331},[101],{"categories":3333},[107,56],{"categories":3335},[107],{"categories":3337},[156],{"categories":3339},[104],{"categories":3341},[149],{"categories":3343},[163],{"categories":3345},[107],{"categories":3347},[],{"categories":3349},[107],{"categories":3351},[104],{"categories":3353},[101],{"categories":3355},[],{"categories":3357},[],{"categories":3359},[104],{"categories":3361},[149],{"categories":3363},[104],{"categories":3365},[],{"categories":3367},[125],{"categories":3369},[],{"categories":3371},[125],{"categories":3373},[156],{"categories":3375},[107],{"categories":3377},[104],{"categories":3379},[163],{"categories":3381},[156],{"categories":3383},[],{"categories":3385},[125],{"categories":3387},[104],{"categories":3389},[],{"categories":3391},[104],{"categories":3393},[107],{"categories":3395},[104],{"categories":3397},[107],{"categories":3399},[104],{"categories":3401},[104],{"categories":3403},[104],{"categories":3405},[104],{"categories":3407},[101],{"categories":3409},[],{"categories":3411},[110],{"categories":3413},[125],{"categories":3415},[104],{"categories":3417},[],{"categories":3419},[156],{"categories":3421},[104],{"categories":3423},[104],{"categories":3425},[107],{"categories":3427},[125],{"categories":3429},[104],{"categories":3431},[104],{"categories":3433},[101],{"categories":3435},[107],{"categories":3437},[146],{"categories":3439},[],{"categories":3441},[149],{"categories":3443},[104],{"categories":3445},[],{"categories":3447},[125],{"categories":3449},[163],{"categories":3451},[],{"categories":3453},[],{"categories":3455},[125],{"categories":3457},[125],{"categories":3459},[163],{"categories":3461},[98],{"categories":3463},[107],{"categories":3465},[107],{"categories":3467},[104],{"categories":3469},[101],{"categories":3471},[],{"categories":3473},[],{"categories":3475},[125],{"categories":3477},[149],{"categories":3479},[156],{"categories":3481},[107],{"categories":3483},[146],{"categories":3485},[149],{"categories":3487},[149],{"categories":3489},[],{"categories":3491},[125],{"categories":3493},[104],{"categories":3495},[104],{"categories":3497},[156],{"categories":3499},[],{"categories":3501},[125],{"categories":3503},[125],{"categories":3505},[125],{"categories":3507},[],{"categories":3509},[107],{"categories":3511},[104],{"categories":3513},[],{"categories":3515},[98],{"categories":3517},[101],{"categories":3519},[],{"categories":3521},[104],{"categories":3523},[104],{"categories":3525},[],{"categories":3527},[156],{"categories":3529},[],{"categories":3531},[],{"categories":3533},[],{"categories":3535},[],{"categories":3537},[104],{"categories":3539},[125],{"categories":3541},[],{"categories":3543},[],{"categories":3545},[104],{"categories":3547},[104],{"categories":3549},[104],{"categories":3551},[149],{"categories":3553},[104],{"categories":3555},[149],{"categories":3557},[],{"categories":3559},[149],{"categories":3561},[149],{"categories":3563},[56],{"categories":3565},[107],{"categories":3567},[156],{"categories":3569},[],{"categories":3571},[],{"categories":3573},[149],{"categories":3575},[156],{"categories":3577},[156],{"categories":3579},[156],{"categories":3581},[],{"categories":3583},[98],{"categories":3585},[156],{"categories":3587},[156],{"categories":3589},[98],{"categories":3591},[156],{"categories":3593},[101],{"categories":3595},[156],{"categories":3597},[156],{"categories":3599},[156],{"categories":3601},[149],{"categories":3603},[125],{"categories":3605},[125],{"categories":3607},[104],{"categories":3609},[156],{"categories":3611},[149],{"categories":3613},[56],{"categories":3615},[149],{"categories":3617},[149],{"categories":3619},[149],{"categories":3621},[],{"categories":3623},[101],{"categories":3625},[],{"categories":3627},[56],{"categories":3629},[156],{"categories":3631},[156],{"categories":3633},[156],{"categories":3635},[107],{"categories":3637},[125,101],{"categories":3639},[149],{"categories":3641},[],{"categories":3643},[],{"categories":3645},[149],{"categories":3647},[],{"categories":3649},[149],{"categories":3651},[125],{"categories":3653},[107],{"categories":3655},[],{"categories":3657},[156],{"categories":3659},[104],{"categories":3661},[146],{"categories":3663},[],{"categories":3665},[104],{"categories":3667},[],{"categories":3669},[125],{"categories":3671},[98],{"categories":3673},[149],{"categories":3675},[],{"categories":3677},[156],{"categories":3679},[125],[3681,3747,3808,3879],{"id":3682,"title":3683,"ai":3684,"body":3689,"categories":3725,"created_at":57,"date_modified":57,"description":49,"extension":58,"faq":57,"featured":59,"kicker_label":57,"meta":3726,"navigation":77,"path":3735,"published_at":3736,"question":57,"scraped_at":3736,"seo":3737,"sitemap":3738,"source_id":3739,"source_name":3740,"source_type":85,"source_url":3741,"stem":3742,"tags":3743,"thumbnail_url":57,"tldr":3744,"tweet":57,"unknown_tags":3745,"__hash__":3746},"summaries\u002Fsummaries\u002Fa2a811b50a4c64f5-mrc-resilient-networking-for-100k-gpu-ai-training-summary.md","MRC: Resilient Networking for 100K+ GPU AI Training",{"provider":7,"model":8,"input_tokens":3685,"output_tokens":3686,"processing_time_ms":3687,"cost_usd":3688},9014,2044,25377,0.0028023,{"type":14,"value":3690,"toc":3719},[3691,3695,3698,3702,3705,3709,3712,3716],[17,3692,3694],{"id":3693},"multi-plane-topologies-slash-switch-tiers-and-power-for-massive-clusters","Multi-Plane Topologies Slash Switch Tiers and Power for Massive Clusters",[22,3696,3697],{},"Traditional 800Gb\u002Fs networks require three or four tiers of switches to connect over 100,000 GPUs, increasing power use, failure points, and cost. MRC splits each 800Gb\u002Fs interface into eight 100Gb\u002Fs links, creating eight parallel 'planes' that connect to separate switches. A 64-port 800Gb\u002Fs switch now handles 512 ports at 100Gb\u002Fs, enabling full connectivity for 131,000 GPUs using only two tiers. This design boosts path diversity—keeping more traffic local to Tier 0 switches—while cutting components, power, and cost compared to single-plane setups. Without changes, single-path flows (like classic RoCE) still congest links as flows collide, especially in AI's collective communications where worst-case latency stalls synchronous training.",[17,3699,3701],{"id":3700},"packet-spraying-and-srv6-eliminate-congestion-and-dynamic-routing","Packet Spraying and SRv6 Eliminate Congestion and Dynamic Routing",[22,3703,3704],{},"MRC sprays packets from a single transfer across hundreds of paths spanning all planes, using final memory addresses for out-of-order reassembly at the destination. Adaptive load-balancing monitors paths: congestion triggers path swaps, packet loss retires the path (with probes for recovery), and 'packet trimming' at switches forwards headers only during destination congestion to prompt retransmits without false failure alarms. This achieves microsecond failure detection and rerouting, versus seconds for traditional fabrics. MRC replaces BGP dynamic routing with static SRv6 source routing: senders embed full switch ID sequences in IPv6 addresses. Switches shift addresses and follow pre-configured static tables, blindly forwarding without recomputing routes. Failures simply retire paths at endpoints, simplifying control planes and eliminating routing bugs from switch software.",[17,3706,3708],{"id":3707},"production-impact-zero-measurable-downtime-amid-constant-failures","Production Impact: Zero-Measurable Downtime Amid Constant Failures",[22,3710,3711],{},"In OpenAI's NVIDIA GB200 supercomputers (including OCI's Abilene Stargate site and Microsoft's Fairwater), MRC handles millions of links with frequent flaps—multiple per minute between tiers—yet synchronous pretraining jobs show no measurable impact, allowing deferred repairs. Rebooting four Tier-1 switches or repairing links during jobs requires no coordination; MRC avoids bad paths automatically. Real training data shows quick recovery from full T1 switch loss with temporary slowdowns far less than physical capacity loss (e.g., one failed port on an 8-port interface reduces max rate by 1\u002F8th but sustains better effective throughput via path recalculation). Multi-job clusters avoid inter-job interference due to core-wide congestion elimination, maximizing GPU utilization for frontier models like those powering ChatGPT (900M weekly users).",[17,3713,3715],{"id":3714},"strategic-wins-simpler-stacks-for-stargate-scale-compute","Strategic Wins: Simpler Stacks for Stargate-Scale Compute",[22,3717,3718],{},"MRC delivers three edges: two-tier multi-plane redundancy with lower power; zero core congestion for consistent flow throughput in sync training; and SRv6 for instant failure bypass via static planes. Deployed with AMD, Broadcom, Intel, Microsoft, NVIDIA hardware, it's released via Open Compute Project for industry adoption, supporting OpenAI's compute strategy of shared standards to scale AI infrastructure efficiently.",{"title":49,"searchDepth":50,"depth":50,"links":3720},[3721,3722,3723,3724],{"id":3693,"depth":50,"text":3694},{"id":3700,"depth":50,"text":3701},{"id":3707,"depth":50,"text":3708},{"id":3714,"depth":50,"text":3715},[56],{"content_references":3727,"triage":3733},[3728,3732],{"type":68,"title":3729,"url":3730,"context":3731},"OCP MRC 1.0","https:\u002F\u002Fwww.opencompute.org\u002Fdocuments\u002Focp-mrc-1-0-pdf","mentioned",{"type":63,"title":64,"url":65,"context":3731},{"relevance":73,"novelty":73,"quality":74,"actionability":50,"composite":75,"reasoning":3734},"Category: DevOps & Cloud. The article discusses the MRC protocol's innovative networking solutions for AI training, which could be relevant for those building AI-powered products. However, it lacks direct actionable insights for the audience, focusing more on technical specifications than practical applications.","\u002Fsummaries\u002Fa2a811b50a4c64f5-mrc-resilient-networking-for-100k-gpu-ai-training-summary","2026-05-11 15:04:27",{"title":3683,"description":49},{"loc":3735},"a2a811b50a4c64f5","OpenAI News","https:\u002F\u002Fopenai.com\u002Findex\u002Fmrc-supercomputer-networking","summaries\u002Fa2a811b50a4c64f5-mrc-resilient-networking-for-100k-gpu-ai-training-summary",[89,90,91],"OpenAI's MRC protocol uses multi-plane topologies and packet spraying across hundreds of paths with SRv6 source routing to eliminate congestion, route around failures in microseconds, and connect 131k GPUs with just two switch tiers, enabling non-stop frontier model training.",[],"BYXvfLzxxajQIir95xuUTVdTfvID4wPt3TOVHNxrCSU",{"id":3748,"title":3749,"ai":3750,"body":3755,"categories":3783,"created_at":57,"date_modified":57,"description":49,"extension":58,"faq":57,"featured":59,"kicker_label":57,"meta":3784,"navigation":77,"path":3795,"published_at":3796,"question":57,"scraped_at":3797,"seo":3798,"sitemap":3799,"source_id":3800,"source_name":3801,"source_type":85,"source_url":3802,"stem":3803,"tags":3804,"thumbnail_url":57,"tldr":3805,"tweet":57,"unknown_tags":3806,"__hash__":3807},"summaries\u002Fsummaries\u002Ff78d6045a31221d2-mrc-enables-100k-gpu-clusters-with-resilient-multi-summary.md","MRC Enables 100k+ GPU Clusters with Resilient Multipath Networking",{"provider":7,"model":8,"input_tokens":3751,"output_tokens":3752,"processing_time_ms":3753,"cost_usd":3754},4244,1621,21683,0.00163665,{"type":14,"value":3756,"toc":3778},[3757,3761,3764,3768,3771,3775],[17,3758,3760],{"id":3759},"multipath-routing-fixes-core-bottlenecks-in-ai-training","Multipath Routing Fixes Core Bottlenecks in AI Training",[22,3762,3763],{},"MRC (Multipath Reliable Connection) eliminates congestion in AI supercomputers by distributing packets across hundreds of network paths simultaneously, rather than single paths. This delivers faster, more predictable GPU-to-GPU data transfers critical for training massive models. On failures—links, switches, or paths—MRC reroutes in microseconds, versus seconds or tens of seconds for standard 800 Gb\u002Fs fabrics. Result: Training jobs survive reboots and maintenance without stalls. OpenAI's multi-plane design connects over 100,000 GPUs using only two Ethernet switch tiers, slashing component count, power use, and costs compared to conventional three- or four-tier setups.",[17,3765,3767],{"id":3766},"proven-at-scale-on-frontier-supercomputers","Proven at Scale on Frontier Supercomputers",[22,3769,3770],{},"Deployed across OpenAI's largest NVIDIA GB200 clusters—including Oracle Cloud in Abilene, Texas, and Microsoft's Fairwater—MRC handled a real-world test during frontier model training for ChatGPT and Codex. Four tier-1 switches rebooted without coordinating with running jobs, proving zero-disruption resilience. This lets operators maintain networks mid-training, boosting uptime for trillion-parameter models where network stalls previously cost hours or days.",[17,3772,3774],{"id":3773},"open-standards-accelerate-adoption","Open Standards Accelerate Adoption",[22,3776,3777],{},"Specification released via Open Compute Project (OCP MRC 1.0), with contributions from AMD, Broadcom, Intel, Microsoft, and NVIDIA. Builders can implement now for Ethernet-based AI fabrics, avoiding proprietary lock-in while hitting supercomputer-scale performance.",{"title":49,"searchDepth":50,"depth":50,"links":3779},[3780,3781,3782],{"id":3759,"depth":50,"text":3760},{"id":3766,"depth":50,"text":3767},{"id":3773,"depth":50,"text":3774},[125],{"content_references":3785,"triage":3793},[3786,3788,3790],{"type":63,"title":3787,"url":65,"context":3731},"Resilient AI Supercomputer Networking Using MRC and SRv6",{"type":68,"title":3729,"publisher":3789,"url":3730,"context":3731},"Open Compute Project",{"type":68,"title":3791,"author":3792,"url":70,"context":66},"MRC Supercomputer Networking","OpenAI",{"relevance":73,"novelty":73,"quality":74,"actionability":50,"composite":75,"reasoning":3794},"Category: AI & LLMs. The article discusses a new networking protocol that addresses bottlenecks in AI supercomputing, which is relevant to AI engineering. However, it lacks direct actionable insights for product builders on how to implement or leverage this technology in their own projects.","\u002Fsummaries\u002Ff78d6045a31221d2-mrc-enables-100k-gpu-clusters-with-resilient-multi-summary","2026-05-06 19:13:21","2026-05-07 11:24:04",{"title":3749,"description":49},{"loc":3795},"f78d6045a31221d2","The Decoder","https:\u002F\u002Fthe-decoder.com\u002Fopenai-built-a-networking-protocol-with-amd-broadcom-intel-microsoft-and-nvidia-to-fix-ai-supercomputer-bottlenecks\u002F","summaries\u002Ff78d6045a31221d2-mrc-enables-100k-gpu-clusters-with-resilient-multi-summary",[90,91,89],"OpenAI's MRC protocol spreads packets across hundreds of paths for microsecond failure recovery, connecting 100,000+ GPUs via just 2 switch tiers—cutting power, cost, and downtime in AI training supercomputers.",[],"LvMASfYTesYX0l3RENkA3FOBQpD3T6H-0KnDqYX6HvU",{"id":3809,"title":3810,"ai":3811,"body":3816,"categories":3850,"created_at":57,"date_modified":57,"description":49,"extension":58,"faq":57,"featured":59,"kicker_label":57,"meta":3851,"navigation":77,"path":3866,"published_at":3867,"question":57,"scraped_at":3868,"seo":3869,"sitemap":3870,"source_id":3871,"source_name":3872,"source_type":85,"source_url":3873,"stem":3874,"tags":3875,"thumbnail_url":57,"tldr":3876,"tweet":57,"unknown_tags":3877,"__hash__":3878},"summaries\u002Fsummaries\u002F6ee9b4f709da3a06-tpus-dominate-at-infrastructure-scale-over-per-chi-summary.md","TPUs Dominate at Infrastructure Scale Over Per-Chip GPU Wins",{"provider":7,"model":8,"input_tokens":3812,"output_tokens":3813,"processing_time_ms":3814,"cost_usd":3815},5399,1852,23082,0.00198315,{"type":14,"value":3817,"toc":3845},[3818,3822,3825,3828,3832,3835,3838,3842],[17,3819,3821],{"id":3820},"infrastructure-scaling-trumps-per-chip-performance","Infrastructure Scaling Trumps Per-Chip Performance",[22,3823,3824],{},"Google's TPU v8t for training and v8i for inference trail Nvidia's Rubin and AMD GPUs in raw per-chip compute and memory. However, evaluating at infrastructure level reveals TPUs' edge: Nvidia's NVL72 scales 72 Rubin GPUs per rack, while Google's 4x4x4 cube interconnects up to 9600 TPUs into a superpod delivering 121 exaFLOPS in FP4—surpassing Nvidia's 1152-GPU Rubin pod at 60 exaFLOPS FP4. Google's Virgo network further scales out to 134,000 chips, potentially reaching 1 million, minimizing network overhead via ICI and optical interconnects. This Lego-like modularity avoids the scaling cliffs Nvidia faces when stacking GPUs, where interconnect overhead erodes per-chip advantages.",[22,3826,3827],{},"Nvidia balances scale-out with InfiniBand for diverse customers (neo-clouds like CoreWeave, labs like OpenAI\u002FMeta, hyperscalers like Microsoft\u002FAmazon), prioritizing broad demand profiles. Google, serving internal apps like Gemini and Vertex AI plus external deals (Anthropic's $1B TPU commitment: 40% owned, 60% rented; Meta's multi-billion rental), optimizes purely for its high-volume needs without market fragmentation risks.",[17,3829,3831],{"id":3830},"workload-profiles-dictate-hardware-choices","Workload Profiles Dictate Hardware Choices",[22,3833,3834],{},"AI tasks bifurcate demands: training prioritizes network bandwidth over compute\u002Fmemory, benefiting TPU's topology. Inference splits further—prefill (pink line in SemiAnalysis chart) is compute\u002Fmemory-bound for KV cache parallelization; decode (white line) is bandwidth\u002Flatency-bound for autoregressive token streaming. TPU v8t\u002F8i bifurcation matches this: v8t for training's network focus, v8i for inference's varied needs. Virgo flattens network bottlenecks, challenging Nvidia's inference dominance.",[22,3836,3837],{},"Replicating Google's scaling on Nvidia chips risks inefficiency for its varied clientele, locking into a 'balanced diet' pod architecture over specialized superpods.",[17,3839,3841],{"id":3840},"explosive-demand-drives-economics","Explosive Demand Drives Economics",[22,3843,3844],{},"Epoch AI projects 450+ new pre-trained models by 2030, many exceeding GPT-5's ~66 septillion FLOPs (total math ops for weights). A 9600-TPU superpod could theoretically pretrain GPT-5-scale models in under 7 days at FP4 (realistically 3-4 weeks), but efficiency cliffs emerge from memory, bandwidth, or latency based on scale-up\u002Fout choices. Rising inference\u002Ftraining demand amplifies TPU economics: internal fab control ensures supply for massive token serving, positioning Google against Nvidia as workloads evolve toward bandwidth constraints.",{"title":49,"searchDepth":50,"depth":50,"links":3846},[3847,3848,3849],{"id":3820,"depth":50,"text":3821},{"id":3830,"depth":50,"text":3831},{"id":3840,"depth":50,"text":3841},[125],{"content_references":3852,"triage":3864},[3853,3857,3861],{"type":3854,"title":3855,"url":3856,"context":71},"tool","Mammoth AI","http:\u002F\u002Fmammouth.ai",{"type":3858,"title":3859,"author":3860,"context":66},"report","SemiAnalysis AI Demand Profiles Diagram","SemiAnalysis",{"type":3858,"title":3862,"author":3863,"context":66},"Epoch AI Pre-Trained Models Projection","Epoch AI",{"relevance":73,"novelty":73,"quality":74,"actionability":50,"composite":75,"reasoning":3865},"Category: AI & LLMs. The article discusses the performance of Google's TPUs compared to Nvidia GPUs, which is relevant to AI infrastructure but lacks direct actionable insights for product builders. While it provides some new perspectives on scaling AI workloads, it does not offer specific frameworks or techniques that the audience can implement.","\u002Fsummaries\u002F6ee9b4f709da3a06-tpus-dominate-at-infrastructure-scale-over-per-chi-summary","2026-04-30 02:16:18","2026-05-03 16:52:02",{"title":3810,"description":49},{"loc":3866},"a42442ea33b32f06","Caleb Writes Code","https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=b_KxiTPBIb0","summaries\u002F6ee9b4f709da3a06-tpus-dominate-at-infrastructure-scale-over-per-chi-summary",[89,91,90],"Google's TPU v8t (training) and v8i (inference) lag Nvidia GPUs per chip but deliver superior performance at scale—9600-chip superpods hit 121 exaFLOPS FP4—via cube topology and Virgo networking, optimizing for AI's bandwidth-heavy workloads.",[],"EDdnhIFUjAM7yxc1JlkafqMae9PUuMTF3sbgBpRfDo4",{"id":3880,"title":3881,"ai":3882,"body":3887,"categories":3923,"created_at":57,"date_modified":57,"description":49,"extension":58,"faq":57,"featured":59,"kicker_label":57,"meta":3924,"navigation":77,"path":3929,"published_at":57,"question":57,"scraped_at":3930,"seo":3931,"sitemap":3932,"source_id":3933,"source_name":3934,"source_type":85,"source_url":3935,"stem":3936,"tags":3937,"thumbnail_url":57,"tldr":3938,"tweet":57,"unknown_tags":3939,"__hash__":3940},"summaries\u002Fsummaries\u002Fe36bce4050e60b52-aws-project-rainier-500k-trainium2-chips-power-mas-summary.md","AWS Project Rainier: 500K Trainium2 Chips Power Massive AI Cluster",{"provider":7,"model":8,"input_tokens":3883,"output_tokens":3884,"processing_time_ms":3885,"cost_usd":3886},5329,1723,13351,0.00190465,{"type":14,"value":3888,"toc":3917},[3889,3893,3896,3900,3903,3907,3910,3914],[17,3890,3892],{"id":3891},"unprecedented-scale-and-speed","Unprecedented Scale and Speed",[22,3894,3895],{},"AWS launched Project Rainier, one of the world's largest AI compute clusters, deploying nearly half a million Trainium2 chips through collaborative innovation. This infrastructure went live in record time, enabling Anthropic to expand to over one million chips by end of 2025. Trainium2 chips optimize AI training workloads cost-effectively compared to general-purpose GPUs, providing builders with massive parallel compute for large-scale model development.",[17,3897,3899],{"id":3898},"advanced-hardware-and-architecture","Advanced Hardware and Architecture",[22,3901,3902],{},"The cluster features UltraServers, transitioning from traditional setups to high-density designs packed with Trainium2 chips. This shift supports extreme compute density, allowing AI teams to train models at scales previously limited by hardware constraints—key for production AI pipelines where chip count directly impacts training throughput and model size.",[17,3904,3906],{"id":3905},"reliability-through-full-stack-control","Reliability Through Full-Stack Control",[22,3908,3909],{},"'No room for failure' drives the design: AWS controls the entire stack, from chips to servers, minimizing downtime in mission-critical AI training. Technicians manage deployments with precision, ensuring 99.99%+ uptime for clusters handling petabyte-scale datasets and trillion-parameter models.",[17,3911,3913],{"id":3912},"sustainability-in-hyperscale-ai","Sustainability in Hyperscale AI",[22,3915,3916],{},"Efficiency scales with size—data centers use advanced cooling (visible water pipes) and power optimization to handle the cluster's immense energy draw without proportional environmental impact. Builders gain access to green compute, reducing carbon footprints for AI workloads while maintaining performance.",{"title":49,"searchDepth":50,"depth":50,"links":3918},[3919,3920,3921,3922],{"id":3891,"depth":50,"text":3892},{"id":3898,"depth":50,"text":3899},{"id":3905,"depth":50,"text":3906},{"id":3912,"depth":50,"text":3913},[125],{"content_references":3925,"triage":3926},[],{"relevance":74,"novelty":73,"quality":74,"actionability":73,"composite":3927,"reasoning":3928},3.6,"Category: AI & LLMs. The article discusses AWS's Project Rainier, which directly relates to AI infrastructure and the optimization of AI training workloads, addressing a specific audience pain point regarding production-ready AI features. It provides insights into advanced hardware and architecture but lacks detailed actionable steps for implementation.","\u002Fsummaries\u002Fe36bce4050e60b52-aws-project-rainier-500k-trainium2-chips-power-mas-summary","2026-04-15 15:27:24",{"title":3881,"description":49},{"loc":3929},"e36bce4050e60b52","__oneoff__","https:\u002F\u002Fwww.aboutamazon.com\u002Fnews\u002Faws\u002Faws-project-rainier-ai-trainium-chips-compute-cluster","summaries\u002Fe36bce4050e60b52-aws-project-rainier-500k-trainium2-chips-power-mas-summary",[91,90,89],"AWS activates Project Rainier with nearly 500,000 Trainium2 chips in record time; Anthropic scales to 1M+ chips by 2025, emphasizing reliability, custom stacks, and sustainability.",[],"oSDXXiRy2R7B6HHze65zAiltgXMqu2yTuE8l8HtUHYg"]