[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"summary-11694bb2ea4dab37-train-gpt-2-llm-from-scratch-on-laptop-summary":3,"summaries-facets-categories":410,"summary-related-11694bb2ea4dab37-train-gpt-2-llm-from-scratch-on-laptop-summary":3995},{"id":4,"title":5,"ai":6,"body":13,"categories":363,"created_at":365,"date_modified":365,"description":354,"extension":366,"faq":365,"featured":367,"kicker_label":365,"meta":368,"navigation":392,"path":393,"published_at":394,"question":365,"scraped_at":395,"seo":396,"sitemap":397,"source_id":398,"source_name":399,"source_type":400,"source_url":401,"stem":402,"tags":403,"thumbnail_url":365,"tldr":407,"tweet":365,"unknown_tags":408,"__hash__":409},"summaries\u002Fsummaries\u002F11694bb2ea4dab37-train-gpt-2-llm-from-scratch-on-laptop-summary.md","Train GPT-2 LLM from Scratch on Laptop",{"provider":7,"model":8,"input_tokens":9,"output_tokens":10,"processing_time_ms":11,"cost_usd":12},"openrouter","x-ai\u002Fgrok-4.1-fast",8437,3044,42622,0.0031869,{"type":14,"value":15,"toc":353},"minimark",[16,21,25,28,34,46,50,53,56,97,100,105,108,111,115,118,121,161,164,188,191,226,229,234,237,241,244,246,271,274,279,282,288,292,295,298,309,312,317,321],[17,18,20],"h2",{"id":19},"why-local-llm-training-reveals-core-mechanics","Why Local LLM Training Reveals Core Mechanics",[22,23,24],"p",{},"Training an LLM from scratch locally demystifies the process, showing 80% of what big labs do without cloud-scale resources. Angelos Perivolaropoulos, who leads speech-to-text at ElevenLabs (creators of top benchmark model Scribe v2), emphasizes starting with basics: no pre-trained weights, pure PyTorch. This tiny GPT-2 variant (vocab=65 chars, context=256, 6 layers) trains fast on laptops, exposing tokenizer choices, architecture blocks, and training loops as the real differentiators between models like GPT-3 vs. GPT-4.",[22,26,27],{},"Key principle: Focus on bi-grams (token pairs). Small vocab (65) yields ~4k bi-grams, coverable by Shakespeare dataset; larger (50k like GPT-2) needs trillions of tokens to converge. \"If you have a model with 200,000 tokens, you need 200,000 tokens squared at least data to train from scratch.\"",[29,30,31],"blockquote",{},[22,32,33],{},"\"We're going to work purely on torch... this is like 80% of the way there to create a model from scratch.\"",[22,35,36,37,41,42,45],{},"Prerequisites: Python 3.12, 16GB RAM (scales down), MPS\u002FCUDA\u002FCPU support. Use UV for env: ",[38,39,40],"code",{},"uv sync",". Colab alternative: ",[38,43,44],{},"!pip install torch numpy datasets tiktoken",". Dataset: Shakespeare (tiny text corpus, downloadable via repo).",[17,47,49],{"id":48},"tokenizer-character-level-for-tiny-models","Tokenizer: Character-Level for Tiny Models",[22,51,52],{},"Start here – LLMs process vectors, not text. Character-level tokenizer maps 65 chars (A-Z, a-z, punctuation, space, newline) to integers via simple dict\u002Fenumerate. Converts strings to int tensors; embedding layer maps to vectors (dim=384).",[22,54,55],{},"Steps:",[57,58,59,67,84,94],"ol",{},[60,61,62,63,66],"li",{},"Load data: ",[38,64,65],{},"text = open('input.txt', 'r').read()"," (Shakespeare).",[60,68,69,70,73,74,73,77,73,80,83],{},"Build vocab: ",[38,71,72],{},"chars = sorted(list(set(text)))","; ",[38,75,76],{},"stoi = {ch:i for i,ch in enumerate(chars)}",[38,78,79],{},"itos = {i:ch for i,ch in enumerate(chars)}",[38,81,82],{},"vocab_size = len(chars)",".",[60,85,86,87,90,91,83],{},"Encode: ",[38,88,89],{},"def encode(s): return [stoi[c] for c in s]","; batch via ",[38,92,93],{},"torch.tensor",[60,95,96],{},"Decode: Reverse for output.",[22,98,99],{},"Trade-off: Low vocab trains fast on small data but poor scaling – model struggles with long-range correlations (e.g., 'sky' + 'is' + 'bl' vs. semantic tokens). For code: Falls to chars for rare vars; BPE (train on data patterns like 'for', 'enumerate') better for prod but needs massive data.",[29,101,102],{},[22,103,104],{},"\"Character level because it's much easier to train... 65*65 = 4,225 possible bi-grams... our dataset should include all bi-grams multiple times.\"",[22,106,107],{},"Common mistake: Using full GPT-2 vocab (50k) – embedding table alone ~19M params (3x model size), won't converge. Future-proof: Train BPE tokenizer on your corpus for real LLMs.",[22,109,110],{},"Quality check: Ensure all bi-grams covered; test encode\u002Fdecode round-trip.",[17,112,114],{"id":113},"causal-transformer-stack-simple-blocks","Causal Transformer: Stack Simple Blocks",[22,116,117],{},"GPT-2 base: Decoder-only, causal self-attention. Don't need PhD-level math – implement blocks, learn why via experimentation.",[22,119,120],{},"Core blocks (per layer):",[122,123,124,135,141,151],"ul",{},[60,125,126,130,131,134],{},[127,128,129],"strong",{},"Multi-head self-attention",": Computes token relationships (QKV matrices). Causal mask prevents future peeking: ",[38,132,133],{},"mask = torch.tril(torch.ones(block_size, block_size))",". Heads (e.g., n_head=6) parallelize; concat + proj.",[60,136,137,140],{},[127,138,139],{},"MLP\u002FFeed-forward",": Processes attended features into logits.",[60,142,143,146,147,150],{},[127,144,145],{},"Residuals",": Add input to output (",[38,148,149],{},"x + sublayer(x)",") – gradients flow directly, stabilizes deep stacks.",[60,152,153,156,157,160],{},[127,154,155],{},"LayerNorm",": Normalizes activations pre-sublayer (",[38,158,159],{},"ln(x) * sublayer(ln(x)) + x","); prevents exploding\u002Fvanishing.",[22,162,163],{},"Model params:",[122,165,166,172,177,182],{},[60,167,168,171],{},[38,169,170],{},"n_embd=384"," (embed dim)",[60,173,174],{},[38,175,176],{},"n_head=6",[60,178,179],{},[38,180,181],{},"n_layer=6",[60,183,184,187],{},[38,185,186],{},"block_size=256"," (context)",[22,189,190],{},"Implementation skeleton (PyTorch nn.Module):",[57,192,193,199,205,212,223],{},[60,194,195,196,83],{},"Embed: ",[38,197,198],{},"self.tok_emb = nn.Embedding(vocab_size, n_embd)",[60,200,201,202,83],{},"Pos embed: ",[38,203,204],{},"self.position_embedding_table = nn.Embedding(block_size, n_embd)",[60,206,207,208,211],{},"Layers: Stack ",[38,209,210],{},"TransformerBlock"," (attention + MLP + norms).",[60,213,214,215,218,219,222],{},"Final: ",[38,216,217],{},"ln_f = LayerNorm(n_embd)"," → ",[38,220,221],{},"lm_head = nn.Linear(n_embd, vocab_size)"," (no bias, tie to embed? Optional).",[60,224,225],{},"Forward: Add pos embeds, loop layers, project logits.",[22,227,228],{},"Principle: Stack identical layers; residuals\u002Fnorms enable scaling depth. Big labs optimize attention for 1M+ context (e.g., avoid O(n²) blowup) but base works.",[29,230,231],{},[22,232,233],{},"\"Attention is what makes transformers different... they can attend to previous tokens and understand relationships.\"",[22,235,236],{},"Mistake: No causal mask → cheats by seeing future. Test: Forward pass on sample, check shapes (batch, seq, vocab).",[17,238,240],{"id":239},"training-loop-where-performance-wins","Training Loop: Where Performance Wins",[22,242,243],{},"Pre-training core: Next-token prediction (cross-entropy loss). Smarter loops separate GPT-3\u002F4 (e.g., Gemini 3 → 3.1 doubles benchmarks via tuning).",[22,245,55],{},[57,247,248,255,258,264],{},[60,249,250,251,254],{},"Data: Split train\u002Fval; generate batches ",[38,252,253],{},"get_batch('train')"," → (B,T) ints.",[60,256,257],{},"Optimize: AdamW, lr=1e-3 (warmup? Basic: constant).",[60,259,260,261,83],{},"Loop: ",[38,262,263],{},"for i in range(max_iters): xb,yb = get_batch(); logits,p = model(xb); loss = F.cross_entropy(logits.view(-1,vocab_size), yb.view(-1)); optimizer.zero_grad(); loss.backward(); optimizer.step()",[60,265,266,267,270],{},"Eval: Perplexity on val (",[38,268,269],{},"torch.exp(loss)",").",[22,272,273],{},"Batch size: 4-64 (RAM-limited); steps: 5k+ for convergence. Estimate iters: dataset_tokens \u002F (batch * block_size).",[29,275,276],{},[22,277,278],{},"\"The training loop is generally the most important part... what you use with the same base model makes the big difference.\"",[22,280,281],{},"Trade-off: Small context (256) fast but forgets long deps; crank on bigger GPU.",[22,283,284,285,83],{},"Inference: Simple ",[38,286,287],{},"while True: generate next token via top-k\u002F1 sample",[17,289,291],{"id":290},"hardware-trade-offs-and-extensions","Hardware Trade-offs and Extensions",[22,293,294],{},"Local constraints force smart choices: 16GB RAM → tiny model (millions params). Colab GPUs free for this scale.",[22,296,297],{},"Scaling path:",[122,299,300,303,306],{},[60,301,302],{},"Bigger data\u002FGPU: BPE tokenizer, 16k context.",[60,304,305],{},"Week-long train: Proper LLM.",[60,307,308],{},"Compete: Optimize loss faster.",[22,310,311],{},"No deep theory needed initially: \"I had no clue how transformers worked... you learn as you push through.\"",[29,313,314],{},[22,315,316],{},"\"Transformers have been commoditized... optimizations on the base idea.\"",[17,318,320],{"id":319},"key-takeaways","Key Takeaways",[122,322,323,326,329,332,338,341,344,347,350],{},[60,324,325],{},"Use character-level tokenizer (65 vocab) for tiny local LLMs; covers bi-grams with small data like Shakespeare.",[60,327,328],{},"Implement causal transformer via 4 blocks: attention (masked), MLP, residual, LayerNorm – stack 6 layers.",[60,330,331],{},"Training: Next-token CE loss, AdamW; monitor val perplexity; 5k iters suffices.",[60,333,334,335,337],{},"Start with ",[38,336,40],{},"; test on Colab if no GPU\u002FRAM.",[60,339,340],{},"Trade-off explicitly: Char tok fast\u002Fcheap but unscalable; BPE for prod needs data.",[60,342,343],{},"Fork repo, beat baseline loss – extend to code tokenizer or longer context.",[60,345,346],{},"Embeddings dominate small models; GPT-2 vocab would 3x size.",[60,348,349],{},"Residuals\u002FLayerNorm stabilize; causal mask essential.",[60,351,352],{},"Bi-grams rule data needs: vocab² minimum tokens.",{"title":354,"searchDepth":355,"depth":355,"links":356},"",2,[357,358,359,360,361,362],{"id":19,"depth":355,"text":20},{"id":48,"depth":355,"text":49},{"id":113,"depth":355,"text":114},{"id":239,"depth":355,"text":240},{"id":290,"depth":355,"text":291},{"id":319,"depth":355,"text":320},[364],"AI & LLMs",null,"md",false,{"content_references":369,"triage":387},[370,375,378,382,384],{"type":371,"title":372,"author":373,"context":374},"other","nanoGPT","Andrej Karpathy","mentioned",{"type":376,"title":377,"context":374},"dataset","Shakespeare",{"type":379,"title":380,"context":381},"tool","UV","recommended",{"type":379,"title":383,"context":374},"tiktoken",{"type":379,"title":385,"author":386,"context":374},"Scribe v2","ElevenLabs",{"relevance":388,"novelty":389,"quality":389,"actionability":388,"composite":390,"reasoning":391},5,4,4.55,"Category: AI & LLMs. This article provides a hands-on workshop for training a GPT-2 model from scratch, which directly addresses the audience's need for practical applications in AI engineering. It includes specific steps and code snippets for building a tokenizer and training loop, making it immediately actionable for developers.",true,"\u002Fsummaries\u002F11694bb2ea4dab37-train-gpt-2-llm-from-scratch-on-laptop-summary","2026-05-04 18:30:06","2026-05-05 16:04:36",{"title":5,"description":354},{"loc":393},"45eb198f2256f249","AI Engineer","article","https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=UsB70Tf5zcE","summaries\u002F11694bb2ea4dab37-train-gpt-2-llm-from-scratch-on-laptop-summary",[404,405,406],"llm","python","coding","Hands-on workshop: Build tokenizer, causal transformer, training loop in PyTorch to train tiny GPT-2 on Shakespeare locally (16GB RAM) or Colab – reveals core engineering without cloud.",[],"5Ukfnhm75lyKlN6uDUvolBl8QzKyNJepG6aCp8NwNTQ",[411,414,417,419,422,425,427,429,431,433,435,437,440,442,444,446,448,450,452,454,456,458,461,464,466,468,471,473,475,478,480,482,484,486,488,490,492,494,496,498,500,502,504,506,508,510,512,514,516,518,520,522,524,526,528,530,532,534,536,538,540,542,544,546,548,550,552,554,556,558,560,562,564,566,568,570,572,574,576,578,580,582,584,586,588,590,592,594,596,598,600,602,604,606,608,610,612,614,616,618,620,622,624,626,628,630,632,634,636,638,640,642,644,646,648,650,652,654,656,658,660,662,664,666,668,670,672,674,676,678,680,682,684,686,688,690,692,694,696,698,700,702,704,706,708,710,712,714,716,718,720,722,724,726,728,730,732,735,737,739,741,743,745,747,749,751,753,755,757,759,761,763,765,767,769,771,773,775,777,779,781,783,785,787,789,791,793,795,797,799,801,803,805,807,809,811,813,815,817,819,821,823,825,827,829,831,833,835,837,839,841,843,845,847,849,851,853,855,857,859,861,863,865,867,869,871,873,875,877,879,881,883,885,887,889,891,893,895,897,899,901,903,905,907,909,911,913,915,917,919,921,923,925,927,929,931,933,935,937,939,941,943,945,947,949,951,953,955,957,959,961,963,965,967,969,971,973,975,977,979,981,983,985,987,989,991,993,995,997,999,1001,1003,1005,1007,1009,1011,1013,1015,1017,1019,1021,1023,1025,1027,1029,1031,1033,1035,1037,1039,1041,1043,1045,1047,1049,1051,1053,1055,1057,1059,1061,1063,1065,1067,1069,1071,1073,1075,1077,1079,1081,1083,1085,1087,1089,1091,1093,1095,1097,1099,1101,1103,1105,1107,1109,1111,1113,1115,1117,1119,1121,1123,1125,1127,1129,1131,1133,1135,1137,1139,1141,1143,1145,1147,1149,1151,1153,1155,1157,1159,1161,1163,1165,1167,1169,1171,1173,1175,1177,1179,1181,1183,1185,1187,1189,1191,1193,1195,1197,1199,1201,1203,1205,1207,1209,1211,1213,1215,1217,1219,1221,1223,1225,1227,1229,1231,1233,1235,1237,1239,1241,1243,1245,1247,1249,1251,1253,1255,1257,1259,1261,1263,1265,1267,1269,1271,1273,1275,1277,1279,1281,1283,1285,1287,1289,1291,1293,1295,1297,1299,1301,1303,1305,1307,1309,1311,1313,1315,1317,1319,1321,1323,1325,1327,1329,1331,1333,1335,1337,1339,1341,1343,1345,1347,1349,1351,1353,1355,1357,1359,1361,1363,1365,1367,1369,1371,1373,1375,1377,1379,1381,1383,1385,1387,1389,1391,1393,1395,1397,1399,1401,1403,1405,1407,1409,1411,1413,1415,1417,1419,1421,1423,1425,1427,1429,1431,1433,1435,1437,1439,1441,1443,1445,1447,1449,1451,1453,1455,1457,1459,1461,1463,1465,1467,1469,1471,1473,1475,1477,1479,1481,1483,1485,1487,1489,1491,1493,1495,1497,1499,1501,1503,1505,1507,1509,1511,1513,1515,1517,1519,1521,1523,1525,1527,1529,1531,1533,1535,1537,1539,1541,1543,1545,1547,1549,1551,1553,1555,1557,1559,1561,1563,1565,1567,1569,1571,1573,1575,1577,1579,1581,1583,1585,1587,1589,1591,1593,1595,1597,1599,1601,1603,1605,1607,1609,1611,1613,1615,1617,1619,1621,1623,1625,1627,1629,1631,1633,1635,1637,1639,1641,1643,1645,1647,1649,1651,1653,1655,1657,1659,1661,1663,1665,1667,1669,1671,1673,1675,1677,1679,1681,1683,1685,1687,1689,1691,1693,1695,1697,1699,1701,1703,1705,1707,1709,1711,1713,1715,1717,1719,1721,1723,1725,1727,1729,1731,1733,1735,1737,1739,1741,1743,1745,1747,1749,1751,1753,1755,1757,1759,1761,1763,1765,1767,1769,1771,1773,1775,1777,1779,1781,1783,1785,1787,1789,1791,1793,1795,1797,1799,1801,1803,1805,1807,1809,1811,1813,1815,1817,1819,1821,1823,1825,1827,1829,1831,1833,1835,1837,1839,1841,1843,1845,1847,1849,1851,1853,1855,1857,1859,1861,1863,1865,1867,1869,1871,1873,1875,1877,1879,1881,1883,1885,1887,1889,1891,1893,1895,1897,1899,1901,1903,1905,1907,1909,1911,1913,1915,1917,1919,1921,1923,1925,1927,1929,1931,1933,1935,1937,1939,1941,1943,1945,1947,1949,1951,1953,1955,1957,1959,1961,1963,1965,1967,1969,1971,1973,1975,1977,1979,1981,1983,1985,1987,1989,1991,1993,1995,1997,1999,2001,2003,2005,2007,2009,2011,2013,2015,2017,2019,2021,2023,2025,2027,2029,2031,2033,2035,2037,2039,2041,2043,2045,2047,2049,2051,2053,2055,2057,2059,2061,2063,2065,2067,2069,2071,2073,2075,2077,2079,2081,2083,2085,2087,2089,2091,2093,2095,2097,2099,2101,2103,2105,2107,2109,2111,2113,2115,2117,2119,2121,2123,2125,2127,2129,2131,2133,2135,2137,2139,2141,2143,2145,2147,2149,2151,2153,2155,2157,2159,2161,2163,2165,2167,2169,2171,2173,2175,2177,2179,2181,2183,2185,2187,2189,2191,2193,2195,2197,2199,2201,2203,2205,2207,2209,2211,2213,2215,2217,2219,2221,2223,2225,2227,2229,2231,2233,2235,2237,2239,2241,2243,2245,2247,2249,2251,2253,2255,2257,2259,2261,2263,2265,2267,2269,2271,2273,2275,2277,2279,2281,2283,2285,2287,2289,2291,2293,2295,2297,2299,2301,2303,2305,2307,2309,2311,2313,2315,2317,2319,2321,2323,2325,2327,2329,2331,2333,2335,2337,2339,2341,2343,2345,2347,2349,2351,2353,2355,2357,2359,2361,2363,2365,2367,2369,2371,2373,2375,2377,2379,2381,2383,2385,2387,2389,2391,2393,2395,2397,2399,2401,2403,2405,2407,2409,2411,2413,2415,2417,2419,2421,2423,2425,2427,2429,2431,2433,2435,2437,2439,2441,2443,2445,2447,2449,2451,2453,2455,2457,2459,2461,2463,2465,2467,2469,2471,2473,2475,2477,2479,2481,2483,2485,2487,2489,2491,2493,2495,2497,2499,2501,2503,2505,2507,2509,2511,2513,2515,2517,2519,2521,2523,2525,2527,2529,2531,2533,2535,2537,2539,2541,2543,2545,2547,2549,2551,2553,2555,2557,2559,2561,2563,2565,2567,2569,2571,2573,2575,2577,2579,2581,2583,2585,2587,2589,2591,2593,2595,2597,2599,2601,2603,2605,2607,2609,2611,2613,2615,2617,2619,2621,2623,2625,2627,2629,2631,2633,2635,2637,2639,2641,2643,2645,2647,2649,2651,2653,2655,2657,2659,2661,2663,2665,2667,2669,2671,2673,2675,2677,2679,2681,2683,2685,2687,2689,2691,2693,2695,2697,2699,2701,2703,2705,2707,2709,2711,2713,2715,2717,2719,2721,2723,2725,2727,2729,2731,2733,2735,2737,2739,2741,2743,2745,2747,2749,2751,2753,2755,2757,2759,2761,2763,2765,2767,2769,2771,2773,2775,2777,2779,2781,2783,2785,2787,2789,2791,2793,2795,2797,2799,2801,2803,2805,2807,2809,2811,2813,2815,2817,2819,2821,2823,2825,2827,2829,2831,2833,2835,2837,2839,2841,2843,2845,2847,2849,2851,2853,2855,2857,2859,2861,2863,2865,2867,2869,2871,2873,2875,2877,2879,2881,2883,2885,2887,2889,2891,2893,2895,2897,2899,2901,2903,2905,2907,2909,2911,2913,2915,2917,2919,2921,2923,2925,2927,2929,2931,2933,2935,2937,2939,2941,2943,2945,2947,2949,2951,2953,2955,2957,2959,2961,2963,2965,2967,2969,2971,2973,2975,2977,2979,2981,2983,2985,2987,2989,2991,2993,2995,2997,2999,3001,3003,3005,3007,3009,3011,3013,3015,3017,3019,3021,3023,3025,3027,3029,3031,3033,3035,3037,3039,3041,3043,3045,3047,3049,3051,3053,3055,3057,3059,3061,3063,3065,3067,3069,3071,3073,3075,3077,3079,3081,3083,3085,3087,3089,3091,3093,3095,3097,3099,3101,3103,3105,3107,3109,3111,3113,3115,3117,3119,3121,3123,3125,3127,3129,3131,3133,3135,3137,3139,3141,3143,3145,3147,3149,3151,3153,3155,3157,3159,3161,3163,3165,3167,3169,3171,3173,3175,3177,3179,3181,3183,3185,3187,3189,3191,3193,3195,3197,3199,3201,3203,3205,3207,3209,3211,3213,3215,3217,3219,3221,3223,3225,3227,3229,3231,3233,3235,3237,3239,3241,3243,3245,3247,3249,3251,3253,3255,3257,3259,3261,3263,3265,3267,3269,3271,3273,3275,3277,3279,3281,3283,3285,3287,3289,3291,3293,3295,3297,3299,3301,3303,3305,3307,3309,3311,3313,3315,3317,3319,3321,3323,3325,3327,3329,3331,3333,3335,3337,3339,3341,3343,3345,3347,3349,3351,3353,3355,3357,3359,3361,3363,3365,3367,3369,3371,3373,3375,3377,3379,3381,3383,3385,3387,3389,3391,3393,3395,3397,3399,3401,3403,3405,3407,3409,3411,3413,3415,3417,3419,3421,3423,3425,3427,3429,3431,3433,3435,3437,3439,3441,3443,3445,3447,3449,3451,3453,3455,3457,3459,3461,3463,3465,3467,3469,3471,3473,3475,3477,3479,3481,3483,3485,3487,3489,3491,3493,3495,3497,3499,3501,3503,3505,3507,3509,3511,3513,3515,3517,3519,3521,3523,3525,3527,3529,3531,3533,3535,3537,3539,3541,3543,3545,3547,3549,3551,3553,3555,3557,3559,3561,3563,3565,3567,3569,3571,3573,3575,3577,3579,3581,3583,3585,3587,3589,3591,3593,3595,3597,3599,3601,3603,3605,3607,3609,3611,3613,3615,3617,3619,3621,3623,3625,3627,3629,3631,3633,3635,3637,3639,3641,3643,3645,3647,3649,3651,3653,3655,3657,3659,3661,3663,3665,3667,3669,3671,3673,3675,3677,3679,3681,3683,3685,3687,3689,3691,3693,3695,3697,3699,3701,3703,3705,3707,3709,3711,3713,3715,3717,3719,3721,3723,3725,3727,3729,3731,3733,3735,3737,3739,3741,3743,3745,3747,3749,3751,3753,3755,3757,3759,3761,3763,3765,3767,3769,3771,3773,3775,3777,3779,3781,3783,3785,3787,3789,3791,3793,3795,3797,3799,3801,3803,3805,3807,3809,3811,3813,3815,3817,3819,3821,3823,3825,3827,3829,3831,3833,3835,3837,3839,3841,3843,3845,3847,3849,3851,3853,3855,3857,3859,3861,3863,3865,3867,3869,3871,3873,3875,3877,3879,3881,3883,3885,3887,3889,3891,3893,3895,3897,3899,3901,3903,3905,3907,3909,3911,3913,3915,3917,3919,3921,3923,3925,3927,3929,3931,3933,3935,3937,3939,3941,3943,3945,3947,3949,3951,3953,3955,3957,3959,3961,3963,3965,3967,3969,3971,3973,3975,3977,3979,3981,3983,3985,3987,3989,3991,3993],{"categories":412},[413],"Developer Productivity",{"categories":415},[416],"Business & SaaS",{"categories":418},[364],{"categories":420},[421],"AI Automation",{"categories":423},[424],"Product Strategy",{"categories":426},[364],{"categories":428},[413],{"categories":430},[416],{"categories":432},[],{"categories":434},[364],{"categories":436},[],{"categories":438},[439],"AI News & Trends",{"categories":441},[421],{"categories":443},[439],{"categories":445},[421],{"categories":447},[421],{"categories":449},[364],{"categories":451},[364],{"categories":453},[439],{"categories":455},[364],{"categories":457},[],{"categories":459},[460],"Design & Frontend",{"categories":462},[463],"Data Science & Visualization",{"categories":465},[439],{"categories":467},[],{"categories":469},[470],"Software Engineering",{"categories":472},[364],{"categories":474},[421],{"categories":476},[477],"Marketing & Growth",{"categories":479},[364],{"categories":481},[421],{"categories":483},[],{"categories":485},[],{"categories":487},[460],{"categories":489},[421],{"categories":491},[413],{"categories":493},[460],{"categories":495},[364],{"categories":497},[421],{"categories":499},[439],{"categories":501},[],{"categories":503},[],{"categories":505},[421],{"categories":507},[470],{"categories":509},[],{"categories":511},[416],{"categories":513},[],{"categories":515},[],{"categories":517},[421],{"categories":519},[421],{"categories":521},[364],{"categories":523},[],{"categories":525},[470],{"categories":527},[],{"categories":529},[],{"categories":531},[],{"categories":533},[364],{"categories":535},[477],{"categories":537},[460],{"categories":539},[460],{"categories":541},[364],{"categories":543},[421],{"categories":545},[364],{"categories":547},[364],{"categories":549},[421],{"categories":551},[421],{"categories":553},[463],{"categories":555},[439],{"categories":557},[421],{"categories":559},[477],{"categories":561},[421],{"categories":563},[424],{"categories":565},[],{"categories":567},[421],{"categories":569},[],{"categories":571},[421],{"categories":573},[470],{"categories":575},[460],{"categories":577},[364],{"categories":579},[],{"categories":581},[],{"categories":583},[421],{"categories":585},[],{"categories":587},[364],{"categories":589},[],{"categories":591},[413],{"categories":593},[470],{"categories":595},[416],{"categories":597},[439],{"categories":599},[364],{"categories":601},[],{"categories":603},[364],{"categories":605},[],{"categories":607},[470],{"categories":609},[463],{"categories":611},[],{"categories":613},[364],{"categories":615},[460],{"categories":617},[],{"categories":619},[460],{"categories":621},[421],{"categories":623},[],{"categories":625},[421],{"categories":627},[439],{"categories":629},[416],{"categories":631},[364],{"categories":633},[],{"categories":635},[421],{"categories":637},[364],{"categories":639},[424],{"categories":641},[],{"categories":643},[364],{"categories":645},[421],{"categories":647},[421],{"categories":649},[],{"categories":651},[463],{"categories":653},[364],{"categories":655},[],{"categories":657},[413],{"categories":659},[416],{"categories":661},[364],{"categories":663},[421],{"categories":665},[470],{"categories":667},[364],{"categories":669},[],{"categories":671},[],{"categories":673},[364],{"categories":675},[],{"categories":677},[460],{"categories":679},[],{"categories":681},[364],{"categories":683},[],{"categories":685},[421],{"categories":687},[364],{"categories":689},[460],{"categories":691},[],{"categories":693},[364],{"categories":695},[364],{"categories":697},[416],{"categories":699},[421],{"categories":701},[364],{"categories":703},[460],{"categories":705},[421],{"categories":707},[],{"categories":709},[],{"categories":711},[439],{"categories":713},[],{"categories":715},[364],{"categories":717},[416,477],{"categories":719},[],{"categories":721},[364],{"categories":723},[],{"categories":725},[],{"categories":727},[364],{"categories":729},[],{"categories":731},[364],{"categories":733},[734],"DevOps & Cloud",{"categories":736},[],{"categories":738},[439],{"categories":740},[460],{"categories":742},[],{"categories":744},[439],{"categories":746},[439],{"categories":748},[364],{"categories":750},[477],{"categories":752},[],{"categories":754},[416],{"categories":756},[],{"categories":758},[364,734],{"categories":760},[364],{"categories":762},[364],{"categories":764},[421],{"categories":766},[364,470],{"categories":768},[463],{"categories":770},[364],{"categories":772},[477],{"categories":774},[421],{"categories":776},[421],{"categories":778},[],{"categories":780},[421],{"categories":782},[364,416],{"categories":784},[],{"categories":786},[460],{"categories":788},[460],{"categories":790},[],{"categories":792},[],{"categories":794},[439],{"categories":796},[],{"categories":798},[413],{"categories":800},[470],{"categories":802},[364],{"categories":804},[460],{"categories":806},[421],{"categories":808},[470],{"categories":810},[439],{"categories":812},[460],{"categories":814},[],{"categories":816},[364],{"categories":818},[364],{"categories":820},[364],{"categories":822},[439],{"categories":824},[413],{"categories":826},[364],{"categories":828},[421],{"categories":830},[734],{"categories":832},[460],{"categories":834},[421],{"categories":836},[],{"categories":838},[],{"categories":840},[460],{"categories":842},[439],{"categories":844},[463],{"categories":846},[],{"categories":848},[364],{"categories":850},[364],{"categories":852},[416],{"categories":854},[364],{"categories":856},[364],{"categories":858},[439],{"categories":860},[],{"categories":862},[421],{"categories":864},[470],{"categories":866},[],{"categories":868},[364],{"categories":870},[364],{"categories":872},[421],{"categories":874},[],{"categories":876},[],{"categories":878},[364],{"categories":880},[],{"categories":882},[416],{"categories":884},[421],{"categories":886},[],{"categories":888},[413],{"categories":890},[364],{"categories":892},[416],{"categories":894},[439],{"categories":896},[],{"categories":898},[],{"categories":900},[],{"categories":902},[439],{"categories":904},[439],{"categories":906},[],{"categories":908},[],{"categories":910},[416],{"categories":912},[],{"categories":914},[],{"categories":916},[413],{"categories":918},[],{"categories":920},[477],{"categories":922},[421],{"categories":924},[416],{"categories":926},[421],{"categories":928},[470],{"categories":930},[],{"categories":932},[424],{"categories":934},[460],{"categories":936},[470],{"categories":938},[364],{"categories":940},[421],{"categories":942},[416],{"categories":944},[364],{"categories":946},[],{"categories":948},[],{"categories":950},[470],{"categories":952},[463],{"categories":954},[424],{"categories":956},[421],{"categories":958},[364],{"categories":960},[],{"categories":962},[734],{"categories":964},[],{"categories":966},[421],{"categories":968},[],{"categories":970},[],{"categories":972},[364],{"categories":974},[460],{"categories":976},[477],{"categories":978},[421],{"categories":980},[],{"categories":982},[413],{"categories":984},[],{"categories":986},[439],{"categories":988},[364,734],{"categories":990},[439],{"categories":992},[364],{"categories":994},[416],{"categories":996},[364],{"categories":998},[],{"categories":1000},[416],{"categories":1002},[],{"categories":1004},[470],{"categories":1006},[460],{"categories":1008},[439],{"categories":1010},[463],{"categories":1012},[413],{"categories":1014},[364],{"categories":1016},[470],{"categories":1018},[],{"categories":1020},[],{"categories":1022},[424],{"categories":1024},[],{"categories":1026},[364],{"categories":1028},[],{"categories":1030},[460],{"categories":1032},[460],{"categories":1034},[460],{"categories":1036},[],{"categories":1038},[],{"categories":1040},[439],{"categories":1042},[421],{"categories":1044},[364],{"categories":1046},[364],{"categories":1048},[364],{"categories":1050},[416],{"categories":1052},[364],{"categories":1054},[],{"categories":1056},[470],{"categories":1058},[470],{"categories":1060},[416],{"categories":1062},[],{"categories":1064},[364],{"categories":1066},[364],{"categories":1068},[416],{"categories":1070},[439],{"categories":1072},[477],{"categories":1074},[421],{"categories":1076},[],{"categories":1078},[460],{"categories":1080},[],{"categories":1082},[364],{"categories":1084},[],{"categories":1086},[416],{"categories":1088},[421],{"categories":1090},[],{"categories":1092},[734],{"categories":1094},[463],{"categories":1096},[470],{"categories":1098},[477],{"categories":1100},[470],{"categories":1102},[421],{"categories":1104},[],{"categories":1106},[],{"categories":1108},[421],{"categories":1110},[413],{"categories":1112},[421],{"categories":1114},[424],{"categories":1116},[416],{"categories":1118},[],{"categories":1120},[364],{"categories":1122},[424],{"categories":1124},[364],{"categories":1126},[364],{"categories":1128},[477],{"categories":1130},[460],{"categories":1132},[421],{"categories":1134},[],{"categories":1136},[],{"categories":1138},[734],{"categories":1140},[470],{"categories":1142},[],{"categories":1144},[421],{"categories":1146},[364],{"categories":1148},[460,364],{"categories":1150},[413],{"categories":1152},[],{"categories":1154},[364],{"categories":1156},[413],{"categories":1158},[460],{"categories":1160},[421],{"categories":1162},[470],{"categories":1164},[],{"categories":1166},[364],{"categories":1168},[],{"categories":1170},[413],{"categories":1172},[],{"categories":1174},[421],{"categories":1176},[424],{"categories":1178},[364],{"categories":1180},[364],{"categories":1182},[460],{"categories":1184},[421],{"categories":1186},[734],{"categories":1188},[460],{"categories":1190},[421],{"categories":1192},[364],{"categories":1194},[364],{"categories":1196},[364],{"categories":1198},[439],{"categories":1200},[],{"categories":1202},[424],{"categories":1204},[421],{"categories":1206},[460],{"categories":1208},[421],{"categories":1210},[470],{"categories":1212},[460],{"categories":1214},[421],{"categories":1216},[439],{"categories":1218},[],{"categories":1220},[364],{"categories":1222},[460],{"categories":1224},[364],{"categories":1226},[413],{"categories":1228},[439],{"categories":1230},[364],{"categories":1232},[477],{"categories":1234},[364],{"categories":1236},[364],{"categories":1238},[421],{"categories":1240},[421],{"categories":1242},[364],{"categories":1244},[421],{"categories":1246},[460],{"categories":1248},[364],{"categories":1250},[],{"categories":1252},[],{"categories":1254},[470],{"categories":1256},[],{"categories":1258},[413],{"categories":1260},[734],{"categories":1262},[],{"categories":1264},[413],{"categories":1266},[416],{"categories":1268},[477],{"categories":1270},[],{"categories":1272},[416],{"categories":1274},[],{"categories":1276},[],{"categories":1278},[],{"categories":1280},[],{"categories":1282},[],{"categories":1284},[364],{"categories":1286},[421],{"categories":1288},[734],{"categories":1290},[413],{"categories":1292},[364],{"categories":1294},[470],{"categories":1296},[424],{"categories":1298},[364],{"categories":1300},[477],{"categories":1302},[364],{"categories":1304},[364],{"categories":1306},[364],{"categories":1308},[364,413],{"categories":1310},[470],{"categories":1312},[470],{"categories":1314},[460],{"categories":1316},[364],{"categories":1318},[],{"categories":1320},[],{"categories":1322},[],{"categories":1324},[470],{"categories":1326},[463],{"categories":1328},[439],{"categories":1330},[460],{"categories":1332},[],{"categories":1334},[364],{"categories":1336},[364],{"categories":1338},[],{"categories":1340},[],{"categories":1342},[421],{"categories":1344},[364],{"categories":1346},[416],{"categories":1348},[],{"categories":1350},[413],{"categories":1352},[364],{"categories":1354},[413],{"categories":1356},[364],{"categories":1358},[470],{"categories":1360},[477],{"categories":1362},[364,460],{"categories":1364},[439],{"categories":1366},[460],{"categories":1368},[],{"categories":1370},[734],{"categories":1372},[460],{"categories":1374},[421],{"categories":1376},[],{"categories":1378},[],{"categories":1380},[],{"categories":1382},[],{"categories":1384},[470],{"categories":1386},[421],{"categories":1388},[421],{"categories":1390},[734],{"categories":1392},[364],{"categories":1394},[364],{"categories":1396},[364],{"categories":1398},[],{"categories":1400},[460],{"categories":1402},[],{"categories":1404},[],{"categories":1406},[421],{"categories":1408},[],{"categories":1410},[],{"categories":1412},[477],{"categories":1414},[477],{"categories":1416},[421],{"categories":1418},[],{"categories":1420},[364],{"categories":1422},[364],{"categories":1424},[470],{"categories":1426},[460],{"categories":1428},[460],{"categories":1430},[421],{"categories":1432},[413],{"categories":1434},[364],{"categories":1436},[460],{"categories":1438},[460],{"categories":1440},[421],{"categories":1442},[421],{"categories":1444},[364],{"categories":1446},[],{"categories":1448},[],{"categories":1450},[364],{"categories":1452},[421],{"categories":1454},[439],{"categories":1456},[470],{"categories":1458},[413],{"categories":1460},[364],{"categories":1462},[],{"categories":1464},[421],{"categories":1466},[421],{"categories":1468},[],{"categories":1470},[413],{"categories":1472},[364],{"categories":1474},[413],{"categories":1476},[413],{"categories":1478},[],{"categories":1480},[],{"categories":1482},[421],{"categories":1484},[421],{"categories":1486},[364],{"categories":1488},[364],{"categories":1490},[439],{"categories":1492},[463],{"categories":1494},[424],{"categories":1496},[439],{"categories":1498},[460],{"categories":1500},[],{"categories":1502},[439],{"categories":1504},[],{"categories":1506},[],{"categories":1508},[],{"categories":1510},[],{"categories":1512},[470],{"categories":1514},[463],{"categories":1516},[],{"categories":1518},[364],{"categories":1520},[364],{"categories":1522},[463],{"categories":1524},[470],{"categories":1526},[],{"categories":1528},[],{"categories":1530},[421],{"categories":1532},[439],{"categories":1534},[439],{"categories":1536},[421],{"categories":1538},[413],{"categories":1540},[364,734],{"categories":1542},[],{"categories":1544},[460],{"categories":1546},[413],{"categories":1548},[421],{"categories":1550},[460],{"categories":1552},[],{"categories":1554},[421],{"categories":1556},[421],{"categories":1558},[364],{"categories":1560},[477],{"categories":1562},[470],{"categories":1564},[460],{"categories":1566},[],{"categories":1568},[421],{"categories":1570},[364],{"categories":1572},[421],{"categories":1574},[421],{"categories":1576},[421],{"categories":1578},[477],{"categories":1580},[421],{"categories":1582},[364],{"categories":1584},[],{"categories":1586},[477],{"categories":1588},[439],{"categories":1590},[421],{"categories":1592},[],{"categories":1594},[],{"categories":1596},[364],{"categories":1598},[421],{"categories":1600},[439],{"categories":1602},[421],{"categories":1604},[],{"categories":1606},[],{"categories":1608},[],{"categories":1610},[421],{"categories":1612},[],{"categories":1614},[],{"categories":1616},[463],{"categories":1618},[364],{"categories":1620},[463],{"categories":1622},[439],{"categories":1624},[364],{"categories":1626},[364],{"categories":1628},[421],{"categories":1630},[364],{"categories":1632},[],{"categories":1634},[],{"categories":1636},[734],{"categories":1638},[],{"categories":1640},[],{"categories":1642},[413],{"categories":1644},[],{"categories":1646},[],{"categories":1648},[],{"categories":1650},[],{"categories":1652},[470],{"categories":1654},[439],{"categories":1656},[477],{"categories":1658},[416],{"categories":1660},[364],{"categories":1662},[364],{"categories":1664},[416],{"categories":1666},[],{"categories":1668},[460],{"categories":1670},[421],{"categories":1672},[416],{"categories":1674},[364],{"categories":1676},[364],{"categories":1678},[413],{"categories":1680},[],{"categories":1682},[413],{"categories":1684},[364],{"categories":1686},[477],{"categories":1688},[421],{"categories":1690},[439],{"categories":1692},[416],{"categories":1694},[364],{"categories":1696},[421],{"categories":1698},[],{"categories":1700},[364],{"categories":1702},[413],{"categories":1704},[364],{"categories":1706},[],{"categories":1708},[439],{"categories":1710},[364],{"categories":1712},[],{"categories":1714},[416],{"categories":1716},[364],{"categories":1718},[],{"categories":1720},[],{"categories":1722},[],{"categories":1724},[364],{"categories":1726},[],{"categories":1728},[734],{"categories":1730},[364],{"categories":1732},[],{"categories":1734},[364],{"categories":1736},[364],{"categories":1738},[364],{"categories":1740},[364,734],{"categories":1742},[364],{"categories":1744},[364],{"categories":1746},[460],{"categories":1748},[421],{"categories":1750},[],{"categories":1752},[421],{"categories":1754},[364],{"categories":1756},[364],{"categories":1758},[364],{"categories":1760},[413],{"categories":1762},[413],{"categories":1764},[470],{"categories":1766},[460],{"categories":1768},[421],{"categories":1770},[],{"categories":1772},[364],{"categories":1774},[439],{"categories":1776},[364],{"categories":1778},[416],{"categories":1780},[],{"categories":1782},[734],{"categories":1784},[460],{"categories":1786},[460],{"categories":1788},[421],{"categories":1790},[439],{"categories":1792},[421],{"categories":1794},[364],{"categories":1796},[],{"categories":1798},[364],{"categories":1800},[],{"categories":1802},[],{"categories":1804},[364],{"categories":1806},[364],{"categories":1808},[364],{"categories":1810},[421],{"categories":1812},[364],{"categories":1814},[],{"categories":1816},[463],{"categories":1818},[421],{"categories":1820},[],{"categories":1822},[],{"categories":1824},[364],{"categories":1826},[439],{"categories":1828},[],{"categories":1830},[460],{"categories":1832},[734],{"categories":1834},[439],{"categories":1836},[470],{"categories":1838},[470],{"categories":1840},[439],{"categories":1842},[439],{"categories":1844},[734],{"categories":1846},[],{"categories":1848},[439],{"categories":1850},[364],{"categories":1852},[413],{"categories":1854},[439],{"categories":1856},[],{"categories":1858},[463],{"categories":1860},[439],{"categories":1862},[470],{"categories":1864},[439],{"categories":1866},[734],{"categories":1868},[364],{"categories":1870},[364],{"categories":1872},[],{"categories":1874},[416],{"categories":1876},[],{"categories":1878},[],{"categories":1880},[364],{"categories":1882},[364],{"categories":1884},[364],{"categories":1886},[364],{"categories":1888},[],{"categories":1890},[463],{"categories":1892},[413],{"categories":1894},[],{"categories":1896},[364],{"categories":1898},[364],{"categories":1900},[734],{"categories":1902},[734],{"categories":1904},[],{"categories":1906},[421],{"categories":1908},[439],{"categories":1910},[439],{"categories":1912},[364],{"categories":1914},[421],{"categories":1916},[],{"categories":1918},[460],{"categories":1920},[364],{"categories":1922},[364],{"categories":1924},[],{"categories":1926},[],{"categories":1928},[734],{"categories":1930},[364],{"categories":1932},[470],{"categories":1934},[416],{"categories":1936},[364],{"categories":1938},[],{"categories":1940},[421],{"categories":1942},[413],{"categories":1944},[413],{"categories":1946},[],{"categories":1948},[364],{"categories":1950},[460],{"categories":1952},[421],{"categories":1954},[],{"categories":1956},[364],{"categories":1958},[364],{"categories":1960},[421],{"categories":1962},[],{"categories":1964},[421],{"categories":1966},[470],{"categories":1968},[],{"categories":1970},[364],{"categories":1972},[],{"categories":1974},[364],{"categories":1976},[],{"categories":1978},[364],{"categories":1980},[364],{"categories":1982},[],{"categories":1984},[364],{"categories":1986},[439],{"categories":1988},[364],{"categories":1990},[364],{"categories":1992},[413],{"categories":1994},[364],{"categories":1996},[439],{"categories":1998},[421],{"categories":2000},[],{"categories":2002},[364],{"categories":2004},[477],{"categories":2006},[],{"categories":2008},[],{"categories":2010},[],{"categories":2012},[413],{"categories":2014},[439],{"categories":2016},[421],{"categories":2018},[364],{"categories":2020},[460],{"categories":2022},[421],{"categories":2024},[],{"categories":2026},[421],{"categories":2028},[],{"categories":2030},[364],{"categories":2032},[421],{"categories":2034},[364],{"categories":2036},[],{"categories":2038},[364],{"categories":2040},[364],{"categories":2042},[439],{"categories":2044},[460],{"categories":2046},[421],{"categories":2048},[460],{"categories":2050},[416],{"categories":2052},[],{"categories":2054},[],{"categories":2056},[364],{"categories":2058},[413],{"categories":2060},[439],{"categories":2062},[],{"categories":2064},[],{"categories":2066},[470],{"categories":2068},[460],{"categories":2070},[],{"categories":2072},[364],{"categories":2074},[],{"categories":2076},[477],{"categories":2078},[364],{"categories":2080},[734],{"categories":2082},[470],{"categories":2084},[],{"categories":2086},[421],{"categories":2088},[364],{"categories":2090},[421],{"categories":2092},[421],{"categories":2094},[364],{"categories":2096},[],{"categories":2098},[413],{"categories":2100},[364],{"categories":2102},[416],{"categories":2104},[470],{"categories":2106},[460],{"categories":2108},[],{"categories":2110},[],{"categories":2112},[],{"categories":2114},[421],{"categories":2116},[460],{"categories":2118},[439],{"categories":2120},[364],{"categories":2122},[439],{"categories":2124},[460],{"categories":2126},[],{"categories":2128},[460],{"categories":2130},[439],{"categories":2132},[416],{"categories":2134},[364],{"categories":2136},[439],{"categories":2138},[477],{"categories":2140},[],{"categories":2142},[],{"categories":2144},[463],{"categories":2146},[364,470],{"categories":2148},[439],{"categories":2150},[364],{"categories":2152},[421],{"categories":2154},[421],{"categories":2156},[364],{"categories":2158},[],{"categories":2160},[470],{"categories":2162},[364],{"categories":2164},[463],{"categories":2166},[421],{"categories":2168},[477],{"categories":2170},[734],{"categories":2172},[],{"categories":2174},[413],{"categories":2176},[421],{"categories":2178},[421],{"categories":2180},[470],{"categories":2182},[364],{"categories":2184},[364],{"categories":2186},[],{"categories":2188},[],{"categories":2190},[],{"categories":2192},[734],{"categories":2194},[439],{"categories":2196},[364],{"categories":2198},[364],{"categories":2200},[364],{"categories":2202},[],{"categories":2204},[463],{"categories":2206},[416],{"categories":2208},[],{"categories":2210},[421],{"categories":2212},[734],{"categories":2214},[],{"categories":2216},[460],{"categories":2218},[460],{"categories":2220},[],{"categories":2222},[470],{"categories":2224},[460],{"categories":2226},[364],{"categories":2228},[],{"categories":2230},[439],{"categories":2232},[364],{"categories":2234},[460],{"categories":2236},[421],{"categories":2238},[439],{"categories":2240},[],{"categories":2242},[421],{"categories":2244},[460],{"categories":2246},[364],{"categories":2248},[],{"categories":2250},[364],{"categories":2252},[364],{"categories":2254},[734],{"categories":2256},[439],{"categories":2258},[463],{"categories":2260},[463],{"categories":2262},[],{"categories":2264},[],{"categories":2266},[],{"categories":2268},[421],{"categories":2270},[470],{"categories":2272},[470],{"categories":2274},[],{"categories":2276},[],{"categories":2278},[364],{"categories":2280},[],{"categories":2282},[421],{"categories":2284},[364],{"categories":2286},[],{"categories":2288},[364],{"categories":2290},[416],{"categories":2292},[364],{"categories":2294},[477],{"categories":2296},[421],{"categories":2298},[364],{"categories":2300},[470],{"categories":2302},[439],{"categories":2304},[421],{"categories":2306},[],{"categories":2308},[439],{"categories":2310},[421],{"categories":2312},[421],{"categories":2314},[],{"categories":2316},[416],{"categories":2318},[421],{"categories":2320},[],{"categories":2322},[364],{"categories":2324},[413],{"categories":2326},[439],{"categories":2328},[734],{"categories":2330},[421],{"categories":2332},[421],{"categories":2334},[413],{"categories":2336},[364],{"categories":2338},[],{"categories":2340},[],{"categories":2342},[460],{"categories":2344},[364,416],{"categories":2346},[],{"categories":2348},[413],{"categories":2350},[463],{"categories":2352},[364],{"categories":2354},[470],{"categories":2356},[364],{"categories":2358},[421],{"categories":2360},[364],{"categories":2362},[364],{"categories":2364},[439],{"categories":2366},[421],{"categories":2368},[],{"categories":2370},[],{"categories":2372},[421],{"categories":2374},[364],{"categories":2376},[734],{"categories":2378},[],{"categories":2380},[364],{"categories":2382},[421],{"categories":2384},[],{"categories":2386},[364],{"categories":2388},[477],{"categories":2390},[463],{"categories":2392},[421],{"categories":2394},[364],{"categories":2396},[734],{"categories":2398},[],{"categories":2400},[364],{"categories":2402},[477],{"categories":2404},[460],{"categories":2406},[364],{"categories":2408},[],{"categories":2410},[477],{"categories":2412},[439],{"categories":2414},[364],{"categories":2416},[364],{"categories":2418},[413],{"categories":2420},[],{"categories":2422},[],{"categories":2424},[460],{"categories":2426},[364],{"categories":2428},[463],{"categories":2430},[477],{"categories":2432},[477],{"categories":2434},[439],{"categories":2436},[],{"categories":2438},[],{"categories":2440},[364],{"categories":2442},[],{"categories":2444},[364,470],{"categories":2446},[439],{"categories":2448},[421],{"categories":2450},[470],{"categories":2452},[364],{"categories":2454},[413],{"categories":2456},[],{"categories":2458},[],{"categories":2460},[413],{"categories":2462},[477],{"categories":2464},[364],{"categories":2466},[],{"categories":2468},[460,364],{"categories":2470},[734],{"categories":2472},[413],{"categories":2474},[],{"categories":2476},[416],{"categories":2478},[416],{"categories":2480},[364],{"categories":2482},[470],{"categories":2484},[421],{"categories":2486},[439],{"categories":2488},[477],{"categories":2490},[460],{"categories":2492},[364],{"categories":2494},[364],{"categories":2496},[364],{"categories":2498},[413],{"categories":2500},[364],{"categories":2502},[421],{"categories":2504},[439],{"categories":2506},[],{"categories":2508},[],{"categories":2510},[463],{"categories":2512},[470],{"categories":2514},[364],{"categories":2516},[460],{"categories":2518},[463],{"categories":2520},[364],{"categories":2522},[364],{"categories":2524},[421],{"categories":2526},[421],{"categories":2528},[364,416],{"categories":2530},[],{"categories":2532},[460],{"categories":2534},[],{"categories":2536},[364],{"categories":2538},[439],{"categories":2540},[413],{"categories":2542},[413],{"categories":2544},[421],{"categories":2546},[364],{"categories":2548},[416],{"categories":2550},[470],{"categories":2552},[477],{"categories":2554},[],{"categories":2556},[439],{"categories":2558},[364],{"categories":2560},[364],{"categories":2562},[439],{"categories":2564},[470],{"categories":2566},[364],{"categories":2568},[421],{"categories":2570},[439],{"categories":2572},[364],{"categories":2574},[460],{"categories":2576},[364],{"categories":2578},[364],{"categories":2580},[734],{"categories":2582},[424],{"categories":2584},[421],{"categories":2586},[364],{"categories":2588},[439],{"categories":2590},[421],{"categories":2592},[477],{"categories":2594},[364],{"categories":2596},[],{"categories":2598},[364],{"categories":2600},[],{"categories":2602},[],{"categories":2604},[],{"categories":2606},[416],{"categories":2608},[364],{"categories":2610},[421],{"categories":2612},[439],{"categories":2614},[439],{"categories":2616},[439],{"categories":2618},[439],{"categories":2620},[],{"categories":2622},[413],{"categories":2624},[421],{"categories":2626},[439],{"categories":2628},[413],{"categories":2630},[421],{"categories":2632},[364],{"categories":2634},[364,421],{"categories":2636},[421],{"categories":2638},[734],{"categories":2640},[439],{"categories":2642},[439],{"categories":2644},[421],{"categories":2646},[364],{"categories":2648},[],{"categories":2650},[439],{"categories":2652},[477],{"categories":2654},[413],{"categories":2656},[364],{"categories":2658},[364],{"categories":2660},[],{"categories":2662},[470],{"categories":2664},[],{"categories":2666},[413],{"categories":2668},[421],{"categories":2670},[439],{"categories":2672},[364],{"categories":2674},[439],{"categories":2676},[413],{"categories":2678},[439],{"categories":2680},[439],{"categories":2682},[],{"categories":2684},[416],{"categories":2686},[421],{"categories":2688},[439],{"categories":2690},[439],{"categories":2692},[439],{"categories":2694},[439],{"categories":2696},[439],{"categories":2698},[439],{"categories":2700},[439],{"categories":2702},[439],{"categories":2704},[439],{"categories":2706},[439],{"categories":2708},[463],{"categories":2710},[413],{"categories":2712},[364],{"categories":2714},[364],{"categories":2716},[],{"categories":2718},[364,413],{"categories":2720},[],{"categories":2722},[421],{"categories":2724},[439],{"categories":2726},[421],{"categories":2728},[364],{"categories":2730},[364],{"categories":2732},[364],{"categories":2734},[364],{"categories":2736},[364],{"categories":2738},[421],{"categories":2740},[416],{"categories":2742},[460],{"categories":2744},[439],{"categories":2746},[364],{"categories":2748},[],{"categories":2750},[],{"categories":2752},[421],{"categories":2754},[460],{"categories":2756},[364],{"categories":2758},[],{"categories":2760},[],{"categories":2762},[477],{"categories":2764},[364],{"categories":2766},[],{"categories":2768},[],{"categories":2770},[413],{"categories":2772},[416],{"categories":2774},[364],{"categories":2776},[416],{"categories":2778},[460],{"categories":2780},[],{"categories":2782},[439],{"categories":2784},[],{"categories":2786},[460],{"categories":2788},[364],{"categories":2790},[477],{"categories":2792},[],{"categories":2794},[477],{"categories":2796},[],{"categories":2798},[],{"categories":2800},[421],{"categories":2802},[],{"categories":2804},[416],{"categories":2806},[413],{"categories":2808},[460],{"categories":2810},[470],{"categories":2812},[],{"categories":2814},[],{"categories":2816},[364],{"categories":2818},[413],{"categories":2820},[477],{"categories":2822},[],{"categories":2824},[421],{"categories":2826},[421],{"categories":2828},[439],{"categories":2830},[364],{"categories":2832},[421],{"categories":2834},[364],{"categories":2836},[421],{"categories":2838},[364],{"categories":2840},[424],{"categories":2842},[439],{"categories":2844},[],{"categories":2846},[477],{"categories":2848},[470],{"categories":2850},[421],{"categories":2852},[],{"categories":2854},[364],{"categories":2856},[421],{"categories":2858},[416],{"categories":2860},[413],{"categories":2862},[364],{"categories":2864},[460],{"categories":2866},[470],{"categories":2868},[470],{"categories":2870},[364],{"categories":2872},[463],{"categories":2874},[364],{"categories":2876},[421],{"categories":2878},[416],{"categories":2880},[421],{"categories":2882},[364],{"categories":2884},[364],{"categories":2886},[421],{"categories":2888},[439],{"categories":2890},[],{"categories":2892},[413],{"categories":2894},[364],{"categories":2896},[421],{"categories":2898},[364],{"categories":2900},[364],{"categories":2902},[],{"categories":2904},[460],{"categories":2906},[416],{"categories":2908},[439],{"categories":2910},[364],{"categories":2912},[364],{"categories":2914},[460],{"categories":2916},[477],{"categories":2918},[463],{"categories":2920},[364],{"categories":2922},[439],{"categories":2924},[364],{"categories":2926},[421],{"categories":2928},[734],{"categories":2930},[364],{"categories":2932},[421],{"categories":2934},[463],{"categories":2936},[],{"categories":2938},[421],{"categories":2940},[470],{"categories":2942},[460],{"categories":2944},[364],{"categories":2946},[413],{"categories":2948},[416],{"categories":2950},[470],{"categories":2952},[],{"categories":2954},[421],{"categories":2956},[364],{"categories":2958},[],{"categories":2960},[439],{"categories":2962},[],{"categories":2964},[439],{"categories":2966},[364],{"categories":2968},[421],{"categories":2970},[421],{"categories":2972},[421],{"categories":2974},[],{"categories":2976},[],{"categories":2978},[364],{"categories":2980},[364],{"categories":2982},[],{"categories":2984},[460],{"categories":2986},[421],{"categories":2988},[477],{"categories":2990},[413],{"categories":2992},[],{"categories":2994},[],{"categories":2996},[439],{"categories":2998},[470],{"categories":3000},[364],{"categories":3002},[364],{"categories":3004},[364],{"categories":3006},[470],{"categories":3008},[439],{"categories":3010},[460],{"categories":3012},[364],{"categories":3014},[364],{"categories":3016},[364],{"categories":3018},[439],{"categories":3020},[364],{"categories":3022},[439],{"categories":3024},[421],{"categories":3026},[421],{"categories":3028},[470],{"categories":3030},[421],{"categories":3032},[364],{"categories":3034},[470],{"categories":3036},[460],{"categories":3038},[],{"categories":3040},[421],{"categories":3042},[],{"categories":3044},[],{"categories":3046},[],{"categories":3048},[416],{"categories":3050},[364],{"categories":3052},[421],{"categories":3054},[413],{"categories":3056},[421],{"categories":3058},[477],{"categories":3060},[],{"categories":3062},[421],{"categories":3064},[],{"categories":3066},[413],{"categories":3068},[421],{"categories":3070},[],{"categories":3072},[421],{"categories":3074},[364],{"categories":3076},[439],{"categories":3078},[364],{"categories":3080},[421],{"categories":3082},[439],{"categories":3084},[421],{"categories":3086},[470],{"categories":3088},[460],{"categories":3090},[413],{"categories":3092},[],{"categories":3094},[421],{"categories":3096},[460],{"categories":3098},[734],{"categories":3100},[439],{"categories":3102},[364],{"categories":3104},[460],{"categories":3106},[413],{"categories":3108},[],{"categories":3110},[421],{"categories":3112},[421],{"categories":3114},[364],{"categories":3116},[],{"categories":3118},[421],{"categories":3120},[424],{"categories":3122},[439],{"categories":3124},[421],{"categories":3126},[416],{"categories":3128},[],{"categories":3130},[364],{"categories":3132},[424],{"categories":3134},[364],{"categories":3136},[421],{"categories":3138},[439],{"categories":3140},[413],{"categories":3142},[734],{"categories":3144},[364],{"categories":3146},[364],{"categories":3148},[364],{"categories":3150},[439],{"categories":3152},[416],{"categories":3154},[364],{"categories":3156},[460],{"categories":3158},[439],{"categories":3160},[734],{"categories":3162},[364],{"categories":3164},[],{"categories":3166},[],{"categories":3168},[734],{"categories":3170},[463],{"categories":3172},[421],{"categories":3174},[421],{"categories":3176},[439],{"categories":3178},[364],{"categories":3180},[413],{"categories":3182},[460],{"categories":3184},[421],{"categories":3186},[364],{"categories":3188},[477],{"categories":3190},[364],{"categories":3192},[421],{"categories":3194},[],{"categories":3196},[364],{"categories":3198},[364],{"categories":3200},[439],{"categories":3202},[413],{"categories":3204},[],{"categories":3206},[364],{"categories":3208},[364],{"categories":3210},[470],{"categories":3212},[460],{"categories":3214},[364,421],{"categories":3216},[477,416],{"categories":3218},[364],{"categories":3220},[],{"categories":3222},[421],{"categories":3224},[],{"categories":3226},[470],{"categories":3228},[364],{"categories":3230},[439],{"categories":3232},[],{"categories":3234},[421],{"categories":3236},[],{"categories":3238},[460],{"categories":3240},[421],{"categories":3242},[413],{"categories":3244},[421],{"categories":3246},[364],{"categories":3248},[734],{"categories":3250},[477],{"categories":3252},[416],{"categories":3254},[416],{"categories":3256},[413],{"categories":3258},[413],{"categories":3260},[364],{"categories":3262},[421],{"categories":3264},[364],{"categories":3266},[364],{"categories":3268},[413],{"categories":3270},[364],{"categories":3272},[477],{"categories":3274},[439],{"categories":3276},[364],{"categories":3278},[421],{"categories":3280},[364],{"categories":3282},[],{"categories":3284},[470],{"categories":3286},[],{"categories":3288},[421],{"categories":3290},[413],{"categories":3292},[],{"categories":3294},[734],{"categories":3296},[364],{"categories":3298},[],{"categories":3300},[439],{"categories":3302},[421],{"categories":3304},[470],{"categories":3306},[364],{"categories":3308},[421],{"categories":3310},[470],{"categories":3312},[421],{"categories":3314},[439],{"categories":3316},[413],{"categories":3318},[439],{"categories":3320},[470],{"categories":3322},[364],{"categories":3324},[460],{"categories":3326},[364],{"categories":3328},[364],{"categories":3330},[364],{"categories":3332},[364],{"categories":3334},[421],{"categories":3336},[364],{"categories":3338},[421],{"categories":3340},[364],{"categories":3342},[413],{"categories":3344},[364],{"categories":3346},[421],{"categories":3348},[460],{"categories":3350},[413],{"categories":3352},[421],{"categories":3354},[460],{"categories":3356},[],{"categories":3358},[364],{"categories":3360},[364],{"categories":3362},[470],{"categories":3364},[],{"categories":3366},[421],{"categories":3368},[477],{"categories":3370},[364],{"categories":3372},[439],{"categories":3374},[477],{"categories":3376},[421],{"categories":3378},[416],{"categories":3380},[416],{"categories":3382},[364],{"categories":3384},[413],{"categories":3386},[],{"categories":3388},[364],{"categories":3390},[],{"categories":3392},[413],{"categories":3394},[364],{"categories":3396},[421],{"categories":3398},[421],{"categories":3400},[],{"categories":3402},[470],{"categories":3404},[470],{"categories":3406},[477],{"categories":3408},[460],{"categories":3410},[],{"categories":3412},[364],{"categories":3414},[413],{"categories":3416},[364],{"categories":3418},[470],{"categories":3420},[413],{"categories":3422},[439],{"categories":3424},[439],{"categories":3426},[],{"categories":3428},[439],{"categories":3430},[421],{"categories":3432},[460],{"categories":3434},[463],{"categories":3436},[364],{"categories":3438},[],{"categories":3440},[439],{"categories":3442},[470],{"categories":3444},[416],{"categories":3446},[364],{"categories":3448},[413],{"categories":3450},[734],{"categories":3452},[413],{"categories":3454},[],{"categories":3456},[],{"categories":3458},[439],{"categories":3460},[],{"categories":3462},[421],{"categories":3464},[421],{"categories":3466},[421],{"categories":3468},[],{"categories":3470},[364],{"categories":3472},[],{"categories":3474},[439],{"categories":3476},[413],{"categories":3478},[460],{"categories":3480},[364],{"categories":3482},[439],{"categories":3484},[439],{"categories":3486},[],{"categories":3488},[439],{"categories":3490},[413],{"categories":3492},[364],{"categories":3494},[],{"categories":3496},[421],{"categories":3498},[421],{"categories":3500},[413],{"categories":3502},[],{"categories":3504},[],{"categories":3506},[],{"categories":3508},[460],{"categories":3510},[421],{"categories":3512},[364],{"categories":3514},[],{"categories":3516},[],{"categories":3518},[],{"categories":3520},[460],{"categories":3522},[],{"categories":3524},[413],{"categories":3526},[],{"categories":3528},[],{"categories":3530},[460],{"categories":3532},[364],{"categories":3534},[439],{"categories":3536},[],{"categories":3538},[477],{"categories":3540},[439],{"categories":3542},[477],{"categories":3544},[364],{"categories":3546},[],{"categories":3548},[],{"categories":3550},[421],{"categories":3552},[],{"categories":3554},[],{"categories":3556},[421],{"categories":3558},[364],{"categories":3560},[],{"categories":3562},[421],{"categories":3564},[439],{"categories":3566},[477],{"categories":3568},[463],{"categories":3570},[421],{"categories":3572},[421],{"categories":3574},[],{"categories":3576},[],{"categories":3578},[],{"categories":3580},[439],{"categories":3582},[],{"categories":3584},[],{"categories":3586},[460],{"categories":3588},[413],{"categories":3590},[],{"categories":3592},[416],{"categories":3594},[477],{"categories":3596},[364],{"categories":3598},[470],{"categories":3600},[413],{"categories":3602},[463],{"categories":3604},[416],{"categories":3606},[470],{"categories":3608},[],{"categories":3610},[],{"categories":3612},[421],{"categories":3614},[413],{"categories":3616},[460],{"categories":3618},[413],{"categories":3620},[421],{"categories":3622},[734],{"categories":3624},[421],{"categories":3626},[],{"categories":3628},[364],{"categories":3630},[439],{"categories":3632},[470],{"categories":3634},[],{"categories":3636},[460],{"categories":3638},[439],{"categories":3640},[413],{"categories":3642},[421],{"categories":3644},[364],{"categories":3646},[416],{"categories":3648},[421,734],{"categories":3650},[421],{"categories":3652},[470],{"categories":3654},[364],{"categories":3656},[463],{"categories":3658},[477],{"categories":3660},[421],{"categories":3662},[],{"categories":3664},[421],{"categories":3666},[364],{"categories":3668},[416],{"categories":3670},[],{"categories":3672},[],{"categories":3674},[364],{"categories":3676},[463],{"categories":3678},[364],{"categories":3680},[],{"categories":3682},[439],{"categories":3684},[],{"categories":3686},[439],{"categories":3688},[470],{"categories":3690},[421],{"categories":3692},[364],{"categories":3694},[477],{"categories":3696},[470],{"categories":3698},[],{"categories":3700},[439],{"categories":3702},[364],{"categories":3704},[],{"categories":3706},[364],{"categories":3708},[421],{"categories":3710},[364],{"categories":3712},[421],{"categories":3714},[364],{"categories":3716},[364],{"categories":3718},[364],{"categories":3720},[364],{"categories":3722},[416],{"categories":3724},[],{"categories":3726},[424],{"categories":3728},[439],{"categories":3730},[364],{"categories":3732},[],{"categories":3734},[470],{"categories":3736},[364],{"categories":3738},[364],{"categories":3740},[421],{"categories":3742},[439],{"categories":3744},[364],{"categories":3746},[364],{"categories":3748},[416],{"categories":3750},[421],{"categories":3752},[460],{"categories":3754},[],{"categories":3756},[463],{"categories":3758},[364],{"categories":3760},[],{"categories":3762},[439],{"categories":3764},[477],{"categories":3766},[],{"categories":3768},[],{"categories":3770},[439],{"categories":3772},[439],{"categories":3774},[477],{"categories":3776},[413],{"categories":3778},[421],{"categories":3780},[421],{"categories":3782},[364],{"categories":3784},[416],{"categories":3786},[],{"categories":3788},[],{"categories":3790},[439],{"categories":3792},[463],{"categories":3794},[470],{"categories":3796},[421],{"categories":3798},[460],{"categories":3800},[463],{"categories":3802},[463],{"categories":3804},[],{"categories":3806},[439],{"categories":3808},[364],{"categories":3810},[364],{"categories":3812},[470],{"categories":3814},[],{"categories":3816},[439],{"categories":3818},[439],{"categories":3820},[439],{"categories":3822},[],{"categories":3824},[421],{"categories":3826},[364],{"categories":3828},[],{"categories":3830},[413],{"categories":3832},[416],{"categories":3834},[],{"categories":3836},[364],{"categories":3838},[364],{"categories":3840},[],{"categories":3842},[470],{"categories":3844},[],{"categories":3846},[],{"categories":3848},[],{"categories":3850},[],{"categories":3852},[364],{"categories":3854},[439],{"categories":3856},[],{"categories":3858},[],{"categories":3860},[364],{"categories":3862},[364],{"categories":3864},[364],{"categories":3866},[463],{"categories":3868},[364],{"categories":3870},[463],{"categories":3872},[],{"categories":3874},[463],{"categories":3876},[463],{"categories":3878},[734],{"categories":3880},[421],{"categories":3882},[470],{"categories":3884},[],{"categories":3886},[],{"categories":3888},[463],{"categories":3890},[470],{"categories":3892},[470],{"categories":3894},[470],{"categories":3896},[],{"categories":3898},[413],{"categories":3900},[470],{"categories":3902},[470],{"categories":3904},[413],{"categories":3906},[470],{"categories":3908},[416],{"categories":3910},[470],{"categories":3912},[470],{"categories":3914},[470],{"categories":3916},[463],{"categories":3918},[439],{"categories":3920},[439],{"categories":3922},[364],{"categories":3924},[470],{"categories":3926},[463],{"categories":3928},[734],{"categories":3930},[463],{"categories":3932},[463],{"categories":3934},[463],{"categories":3936},[],{"categories":3938},[416],{"categories":3940},[],{"categories":3942},[734],{"categories":3944},[470],{"categories":3946},[470],{"categories":3948},[470],{"categories":3950},[421],{"categories":3952},[439,416],{"categories":3954},[463],{"categories":3956},[],{"categories":3958},[],{"categories":3960},[463],{"categories":3962},[],{"categories":3964},[463],{"categories":3966},[439],{"categories":3968},[421],{"categories":3970},[],{"categories":3972},[470],{"categories":3974},[364],{"categories":3976},[460],{"categories":3978},[],{"categories":3980},[364],{"categories":3982},[],{"categories":3984},[439],{"categories":3986},[413],{"categories":3988},[463],{"categories":3990},[],{"categories":3992},[470],{"categories":3994},[439],[3996,4051,4136,4177],{"id":3997,"title":3998,"ai":3999,"body":4004,"categories":4032,"created_at":365,"date_modified":365,"description":354,"extension":366,"faq":365,"featured":367,"kicker_label":365,"meta":4033,"navigation":392,"path":4037,"published_at":4038,"question":365,"scraped_at":4039,"seo":4040,"sitemap":4041,"source_id":4042,"source_name":4043,"source_type":400,"source_url":4044,"stem":4045,"tags":4046,"thumbnail_url":365,"tldr":4048,"tweet":365,"unknown_tags":4049,"__hash__":4050},"summaries\u002Fsummaries\u002F8f8ab2daa22c64d3-collaborative-ai-writer-websockets-crdt-claude-summary.md","Collaborative AI Writer: WebSockets + CRDT + Claude",{"provider":7,"model":8,"input_tokens":4000,"output_tokens":4001,"processing_time_ms":4002,"cost_usd":4003},3901,1273,27480,0.00139405,{"type":14,"value":4005,"toc":4027},[4006,4010,4013,4017,4020,4024],[17,4007,4009],{"id":4008},"bidirectional-real-time-dataflow","Bidirectional Real-Time Dataflow",[22,4011,4012],{},"Clients send edit operations and AI assist requests to a FastAPI WebSocket server, which manages connections per document. The server broadcasts changes bidirectionally: user edits sync instantly across all participants, while Claude's streaming API delivers response tokens one delta at a time to every connected client. This setup supports concurrent editing by multiple users without a central text authority, enabling seamless collaboration where one user types while the AI responds for all.",[17,4014,4016],{"id":4015},"conflict-free-sync-with-per-document-crdt","Conflict-Free Sync with Per-Document CRDT",[22,4018,4019],{},"Use a lightweight CRDT (Conflict-Free Replicated Data Type) scoped to each document to merge concurrent text edits from multiple users without coordination or conflicts. Clients apply operations locally and send them to the server, which replicates the merged state back—ensuring everyone sees the same document version instantly, even during high concurrency.",[17,4021,4023],{"id":4022},"streaming-ai-responses-and-per-user-limits","Streaming AI Responses and Per-User Limits",[22,4025,4026],{},"Integrate Claude's streaming API on the server to generate AI-assisted writing, fanning out each token delta via WebSockets to all room participants as it arrives. Prevent abuse with a token-bucket rate limiter per user, enforcing individual rate limits and cost ceilings so one user's heavy usage doesn't disrupt the shared session. This minimal stack—WebSockets, CRDT, streaming, rate limiting—scales to production without exotic dependencies.",{"title":354,"searchDepth":355,"depth":355,"links":4028},[4029,4030,4031],{"id":4008,"depth":355,"text":4009},{"id":4015,"depth":355,"text":4016},{"id":4022,"depth":355,"text":4023},[364],{"content_references":4034,"triage":4035},[],{"relevance":388,"novelty":389,"quality":389,"actionability":388,"composite":390,"reasoning":4036},"Category: AI & LLMs. The article provides a detailed guide on building a collaborative AI writing tool using WebSockets and CRDTs, addressing practical applications for developers looking to integrate AI into their products. It includes specific technical implementations and frameworks, making it immediately actionable for the target audience.","\u002Fsummaries\u002F8f8ab2daa22c64d3-collaborative-ai-writer-websockets-crdt-claude-summary","2026-05-08 14:52:43","2026-05-09 15:36:31",{"title":3998,"description":354},{"loc":4037},"8f8ab2daa22c64d3","Level Up Coding","https:\u002F\u002Flevelup.gitconnected.com\u002Fbuild-a-real-time-collaborative-ai-writing-tool-with-websockets-and-claude-d34b52fd6095?source=rss----5517fd7b58a6---4","summaries\u002F8f8ab2daa22c64d3-collaborative-ai-writer-websockets-crdt-claude-summary",[404,405,406,4047],"ai-tools","Build multi-user real-time AI writing with FastAPI WebSockets for connections, CRDTs for conflict-free text sync, Claude streaming fanned to all users, and per-user token-bucket rate limiting to avoid bursts.",[],"GW6g8lrW_NwckTbGCOa2C37yTwfTLfmVruA5tfOQ50g",{"id":4052,"title":4053,"ai":4054,"body":4059,"categories":4110,"created_at":365,"date_modified":365,"description":354,"extension":366,"faq":365,"featured":367,"kicker_label":365,"meta":4111,"navigation":392,"path":4122,"published_at":4123,"question":365,"scraped_at":4124,"seo":4125,"sitemap":4126,"source_id":4127,"source_name":4128,"source_type":400,"source_url":4129,"stem":4130,"tags":4131,"thumbnail_url":365,"tldr":4133,"tweet":365,"unknown_tags":4134,"__hash__":4135},"summaries\u002Fsummaries\u002Fadd9ec06f3d8b78d-decoder-only-transformers-drive-gpt-scaling-summary.md","Decoder-Only Transformers Drive GPT Scaling",{"provider":7,"model":8,"input_tokens":4055,"output_tokens":4056,"processing_time_ms":4057,"cost_usd":4058},8457,1685,17671,0.00202705,{"type":14,"value":4060,"toc":4104},[4061,4065,4068,4071,4075,4078,4081,4085,4088,4091,4095,4098,4101],[17,4062,4064],{"id":4063},"self-attention-enables-parallel-long-range-dependencies","Self-Attention Enables Parallel Long-Range Dependencies",[22,4066,4067],{},"Transformers replace RNNs' sequential processing, which suffers vanishing gradients beyond 50-100 words, with self-attention that computes direct relationships between all token pairs simultaneously. For a token like \"it\" in \"The cat sat on the mat and looked at the fishbowl because it was hungry,\" every prior word votes on relevance via query-key dot products scaled by embed_size^{-0.5}, softmax-normalized, and applied to values. This parallelization trains across thousands of GPUs.",[22,4069,4070],{},"GPT's decoder-only design strips away the encoder, applying a causal mask to block future tokens, forcing rich representations solely from predicting the next token. GPT-1 (117M params, 12 layers) showed modest NLP scores, but GPT-2 (1.5B params) gained zero-shot abilities like summarization via prompting. GPT-3 (175B params, 96 layers) added in-context learning from prompt examples without fine-tuning. Deeper layers progress from syntax (early) to reasoning and world models (late). This simplicity scales better than encoder-decoder setups, avoiding cross-attention overhead.",[17,4072,4074],{"id":4073},"moe-and-test-time-compute-scale-beyond-dense-models","MoE and Test-Time Compute Scale Beyond Dense Models",[22,4076,4077],{},"Dense models activate all parameters per token, making trillions unaffordable. Mixture of Experts (MoE) routes each token to 2-8 specialized experts out of 128+, activating ~5% of weights—e.g., DeepSeek-V3 uses 37B active out of 671B total, trained for $5.6M on 2,048 H800 GPUs, matching GPT-4. Multi-Head Latent Attention (MLA) compresses KV cache to cut memory bandwidth. Tradeoffs include expert collapse (router overloads few experts) and full-model memory needs despite sparse activation.",[22,4079,4080],{},"o1 introduced test-time compute: generate internal reasoning chains (30s for hard problems), backtrack dead ends, and refine via RL on verifiable rewards like math solutions. This outperforms larger instant-response models, decoupling ability from size. GPT-5 routes simple queries fast (System 1) and complex ones deeply (System 2). Open models like DeepSeek-R1 replicate this.",[17,4082,4084],{"id":4083},"multimodal-fusion-and-real-world-impacts","Multimodal Fusion and Real-World Impacts",[22,4086,4087],{},"Early fusion embeds vision tokens from Vision Transformers (e.g., MetaCLIP) into the same space as text, enabling unified attention across modalities—no separate captioning. Models like LLaMA 4, Qwen-VL handle charts, 3D spatial reasoning (GLM-4.5V's rotated positional encoding). This yields native cross-modal reasoning, e.g., diagnosing X-rays directly.",[22,4089,4090],{},"Applications: Harvey AI (RAG + fine-tuned GPT-4) cuts legal review 40-60%; GPT-4.1 hits 54.6% on SWE-bench (21.4pp over GPT-4o), ingesting 1M-token codebases; 75% medical accuracy accelerates drug discovery. Open weights (LLaMA, DeepSeek) ensure data sovereignty.",[17,4092,4094],{"id":4093},"implement-mini-gpt-from-scratch-in-pytorch","Implement Mini-GPT from Scratch in PyTorch",[22,4096,4097],{},"Build a character-level GPT: Tokenizer maps unique chars to indices (vocab_size ~50). SelfAttention computes QKV projections, scores = (Q @ K.T) * scale, weights = softmax(scores), out = weights @ V. TransformerBlock adds residual attention + FFN (4x expand, ReLU), LayerNorm post each.",[22,4099,4100],{},"MiniGPT stacks NUM_LAYERS=2 blocks on token + positional embeddings (BLOCK_SIZE=32), outputs logits via linear to vocab_size. Train on dataset.txt: batch BATCH_SIZE=16 sequences, predict next token with CrossEntropyLoss, Adam at 3e-4, 20 EPOCHS. Generation: sample from last-token softmax via multinomial, append up to 100 tokens from context like \"AI is\".",[22,4102,4103],{},"Project structure: data\u002Fdataset.txt, model\u002F{tokenizer,attention,transformer,gpt}.py, train.py saves model.pth, generate.py loads\u002Finfers. Config: EMBED_SIZE=64, NUM_HEADS=4 (implied in attention). This replicates core logic scalably.",{"title":354,"searchDepth":355,"depth":355,"links":4105},[4106,4107,4108,4109],{"id":4063,"depth":355,"text":4064},{"id":4073,"depth":355,"text":4074},{"id":4083,"depth":355,"text":4084},{"id":4093,"depth":355,"text":4094},[364],{"content_references":4112,"triage":4118},[4113],{"type":4114,"title":4115,"author":4116,"context":4117},"paper","Attention Is All You Need","Ashish Vaswani’s team","cited",{"relevance":389,"novelty":4119,"quality":389,"actionability":355,"composite":4120,"reasoning":4121},3,3.4,"Category: AI & LLMs. The article provides a detailed explanation of the architecture behind GPT models, which is relevant for developers looking to integrate AI features. However, while it offers insights into model design, it lacks practical applications or frameworks that the audience can directly implement.","\u002Fsummaries\u002Fadd9ec06f3d8b78d-decoder-only-transformers-drive-gpt-scaling-summary","2026-04-18 19:32:29","2026-04-19 01:22:04",{"title":4053,"description":354},{"loc":4122},"add9ec06f3d8b78d","Python in Plain English","https:\u002F\u002Fpython.plainenglish.io\u002Fthe-architecture-behind-gpt-models-de61992c088a?source=rss----78073def27b8---4","summaries\u002Fadd9ec06f3d8b78d-decoder-only-transformers-drive-gpt-scaling-summary",[404,405,4132,406],"machine-learning","GPT models use decoder-only transformers with causal masking for next-token prediction, enabling emergent zero-shot and in-context learning when scaled massively, now enhanced by MoE for efficiency and reasoning chains.",[],"FYS789V3fqVrHXGVHROYyiskRFH6nT84_QPvh_I63p0",{"id":4137,"title":4138,"ai":4139,"body":4144,"categories":4164,"created_at":365,"date_modified":365,"description":354,"extension":366,"faq":365,"featured":367,"kicker_label":365,"meta":4165,"navigation":392,"path":4166,"published_at":4167,"question":365,"scraped_at":365,"seo":4168,"sitemap":4169,"source_id":4170,"source_name":4128,"source_type":400,"source_url":4171,"stem":4172,"tags":4173,"thumbnail_url":365,"tldr":4174,"tweet":365,"unknown_tags":4175,"__hash__":4176},"summaries\u002Fsummaries\u002Fclaude-sonnet-partially-migrates-python-blog-engin-summary.md","Claude Sonnet Partially Migrates Python Blog Engine to Rust",{"provider":7,"model":8,"input_tokens":4140,"output_tokens":4141,"processing_time_ms":4142,"cost_usd":4143},3666,957,10762,0.0011892,{"type":14,"value":4145,"toc":4160},[4146,4150,4153,4157],[17,4147,4149],{"id":4148},"ai-coding-agents-excel-at-grunt-workwith-limits","AI Coding Agents Excel at Grunt Work—With Limits",[22,4151,4152],{},"AI tools like Claude promise to automate tedious tasks such as porting code between languages, letting humans focus on architecture and review. In practice, this seductive pitch faces real-world stress: migrating a half-finished Python blog engine to Rust required days of 'push-and-pull' interaction with Claude Sonnet, yielding partial success rather than a seamless handoff.",[17,4154,4156],{"id":4155},"real-experiment-reveals-partial-wins-and-breaks","Real Experiment Reveals Partial Wins and Breaks",[22,4158,4159],{},"Senior InfoWorld journalist Serdar Yegulalp, with 30 years in tech, ran an honest test on Claude's ability to handle 'the hardest job in software development'—full language migration. The outcome documented instructive failures and breakthroughs, showing AI agents manage intent description and execution but falter on production-level complexity without heavy human oversight. (Note: Content is truncated teaser; lacks specifics on exact breaks or fixes.)",{"title":354,"searchDepth":355,"depth":355,"links":4161},[4162,4163],{"id":4148,"depth":355,"text":4149},{"id":4155,"depth":355,"text":4156},[364],{},"\u002Fsummaries\u002Fclaude-sonnet-partially-migrates-python-blog-engin-summary","2026-04-08 21:21:20",{"title":4138,"description":354},{"loc":4166},"65670176d72abdc0","https:\u002F\u002Funknown","summaries\u002Fclaude-sonnet-partially-migrates-python-blog-engin-summary",[404,405,4047,406],"InfoWorld's Serdar Yegulalp tested Claude Sonnet on porting a real Python blog engine to Rust over days of iteration; it succeeded partly but exposed limits in handling complex migrations.",[],"ONaQ2FvFXSp_Ykp6gEgbPUI93sQS8sbLwdHmqioBeH8",{"id":4178,"title":4179,"ai":4180,"body":4185,"categories":4237,"created_at":365,"date_modified":365,"description":354,"extension":366,"faq":365,"featured":367,"kicker_label":365,"meta":4238,"navigation":392,"path":4239,"published_at":4240,"question":365,"scraped_at":365,"seo":4241,"sitemap":4242,"source_id":4243,"source_name":4244,"source_type":400,"source_url":4171,"stem":4245,"tags":4246,"thumbnail_url":365,"tldr":4247,"tweet":365,"unknown_tags":4248,"__hash__":4249},"summaries\u002Fsummaries\u002Fmicrogpt-py-full-gpt-in-300-lines-of-pure-python-summary.md","microgpt.py: Full GPT in 300 Lines of Pure Python",{"provider":7,"model":8,"input_tokens":4181,"output_tokens":4182,"processing_time_ms":4183,"cost_usd":4184},11786,1242,8684,0.0029557,{"type":14,"value":4186,"toc":4232},[4187,4191,4206,4210,4217,4221],[17,4188,4190],{"id":4189},"custom-autograd-engine-powers-end-to-end-training","Custom Autograd Engine Powers End-to-End Training",[22,4192,4193,4194,4197,4198,4201,4202,4205],{},"Implements automatic differentiation via ",[38,4195,4196],{},"Value"," class with slots for efficiency. Supports add, mul, pow, log, exp, ReLU, and backward via topological sort on computation graph. Chain rule propagates gradients recursively: ",[38,4199,4200],{},"child.grad += local_grad * v.grad",". Enables full forward\u002Fbackward without libraries. For a names dataset (32k lines from ",[38,4203,4204],{},"names.txt","), builds char-level tokenizer: unique chars (vocab_size=~30+1 BOS token). Model params (~10k total): 1 layer, n_embd=16, block_size=16, n_head=4 (head_dim=4). Weights initialized Gaussian std=0.08. Embeddings: wte (vocab x 16), wpe (16 x 16), lm_head (vocab x 16). Per layer: QKV (4x 16x16), Wo (16x16), MLP fc1 (64x16), fc2 (16x64).",[17,4207,4209],{"id":4208},"gpt-architecture-mirrors-gpt-2-essentials","GPT Architecture Mirrors GPT-2 Essentials",[22,4211,4212,4213,4216],{},"Forward pass: token+pos embeds → RMSNorm → residual blocks. Attention: raw dot-product (scaled by 1\u002Fsqrt(head_dim)), softmax weights → weighted V sum → Wo projection. Causal via key\u002Fvalue history append (no mask). MLP: RMSNorm → fc1 → ReLU → fc2 → residual. Final lm_head logits → softmax probs. Uses RMSNorm (",[38,4214,4215],{},"scale = (mean(x^2)+eps)^-0.5",") over LayerNorm, ReLU over GeLU, no biases. Keys\u002Fvalues persist across positions for KV cache simulation. Loss: average -log P(next_token) over sequence (BOS-wrapped docs, up to block_size=16).",[17,4218,4220],{"id":4219},"adam-training-inference-in-1000-steps","Adam Training + Inference in 1000 Steps",[22,4222,4223,4224,4228,4229,4231],{},"Shuffles 32k names, cycles through docs. Per step: tokenize ",[4225,4226,4227],"span",{},"BOS"," + chars + ",[4225,4230,4227],{},", forward all positions (building KV cache), average cross-entropy loss → backward → Adam update (lr=0.01 linear decay to 0, β1=0.85, β2=0.99). Prints loss (drops from ~3 to ~1.5 typically). Inference: start BOS, sample argmax-probs (temp=0.5) until BOS, yields plausible names like 'korsal' after training. Demonstrates: core GPT is simple; libs optimize speed\u002Fscale. Trade-off: slow (minutes on CPU), but reveals every op.",{"title":354,"searchDepth":355,"depth":355,"links":4233},[4234,4235,4236],{"id":4189,"depth":355,"text":4190},{"id":4208,"depth":355,"text":4209},{"id":4219,"depth":355,"text":4220},[364],{},"\u002Fsummaries\u002Fmicrogpt-py-full-gpt-in-300-lines-of-pure-python-summary","2026-04-08 21:21:19",{"title":4179,"description":354},{"loc":4239},"56d2bdaaa16d5c3b","Andrej Karpathy Gists","summaries\u002Fmicrogpt-py-full-gpt-in-300-lines-of-pure-python-summary",[404,405,4132,406],"Trains a tiny GPT on names dataset using custom autograd—no deps, no PyTorch—to generate realistic names, distilling the core transformer algorithm.",[],"3fO1PHuRnDxVHEXFsDwlj_bugbD79pZ1c6UEJVeKQE8"]