Chinese open-weight MoEs land on HF four times in one week: DeepSeek-V4-Pro + GLM-5.2 + Tencent Hy3 + Meituan LongCat-2.0 — sparse attention at long context is the shared direction

5 sources

In the first week of July, four Chinese open-weight MoE models landed on Hugging Face within one week: DeepSeek-V4-Pro (1.6T total / 49B activated MoE + 1M context + Hybrid Attention CSA+HCA + Muon optimizer + manifold-constrained hyper-connections; DSpark adds a confidence-scheduled speculative decoding module); zai-org/GLM-5.2 (753B MoE + Glm MoE DSA sparse attention + IndexShare 2.9x FLOPs reduction at 1M context); tencent/Hy3 (295B / 21B + 256K + 80 layers + MTP layer); meituan-longcat/LongCat-2.0 (1.6T / ~48B + LongCat Sparse Attention Streaming/Cross-Layer/Hierarchical Indexing + 135B N-gram embedding + 35T+ training tokens). Shared direction: sparse attention at long context.

Why now

In the first week of July, four Chinese open-weight MoEs landed on HF within one week and all four share the same direction — sparse attention at long context. This is the strongest cross-cutting research-side signal of early July; creators can bundle it as 'Chinese open-weight MoE week' with an architecture-comparison card.

Why it is worth publishing

Huge demo surface: each of the four models can be loaded directly from HF and run inference; the sparse-attention architecture comparison (DeepSeek CSA+HCA / GLM IndexShare / Hy3 MTP / LongCat LSA) makes a clean horizontal card.

Evidence basis

All four models concentrated on the HF trending list during the first week of July — as 'Chinese open-weight MoE week' heat is medium-to-high.

“Four Chinese open-weight MoEs landed on Hugging Face this week — and every single one is betting on sparse attention at long context.”

Angle

Frame the first-week-of-July Chinese MoE wave on HF as 'sparse attention at long context is the strongest cross-cutting research-side signal of early July' — lay out the four models by architectural similarity (DeepSeek CSA+HCA / GLM IndexShare / Hy3 MTP / LongCat LSA) rather than reading each card in isolation.

Format

Carousel

Demo idea

Build a 'Chinese open-weight MoE week' architecture card: 4 columns (DeepSeek-V4-Pro / GLM-5.2 / Hy3 / LongCat-2.0), 4 rows (total params / activated params / context length / attention mechanism); fill in each cell with the concrete number + a one-sentence attention-mechanism description.

Platform notes

All four model benchmarks are self-reported on the model cards (medium risk) — don't paraphrase as third-party validation; each model's specific long-context sparse-attention spec is incomplete in the captured summaries (medium risk) — don't fill in numbers from memory.

Usable claims

Hugging Face's deepseek-ai/DeepSeek-V4-Pro-DSpark model card (uploaded 2026-06-27, last modified 2026-07-08) documents DeepSeek-V4-Pro as a 1.6T total / 49B activated MoE with 1M context, MIT license, using a Hybrid Attention design (CSA+HCA) plus the Muon optimizer and manifold-constrained hyper-connections; DSpark adds a confidence-scheduled speculative decoding module with semi-autoregressive generation.
Hugging Face's zai-org/GLM-5.2 model card (uploaded 2026-06-16, last modified 2026-07-02) documents a 753B-parameter MoE (BF16 + F32, sharded safetensors) with Glm MoE DSA — a sparse attention mechanism using 'IndexShare' that reuses one indexer across every four sparse attention layers, claiming 2.9x per-token FLOPs reduction at 1M context; MIT license, no regional restrictions; self-reported benchmark scores include GPQA-Diamond 91.2, SWE-bench Pro 62.1, DeepSWE 46.2, HLE default 40.5 / HLE with tools 54.7, AIME 2026 99.2, Terminal Bench 2.1 (Best Harness) 82.7.
Hugging Face's tencent/Hy3 model card (uploaded 2026-07-02, last modified 2026-07-06) documents Tencent Hunyuan third-generation MoE: 295B total / 21B activated, Apache-2.0 license, 256K context, 80 layers, with a multi-token prediction (MTP) layer; self-reported benchmark scores include GPQA Diamond 90.4, SWE-bench Verified 78, SWE-bench Pro 57.9.
Hugging Face's meituan-longcat/LongCat-2.0 model card (uploaded 2026-07-05, last modified 2026-07-08) documents Meituan LongCat second generation as a 1.6T total / ~48B activated MoE, MIT license, introducing LongCat Sparse Attention (Streaming / Cross-Layer / Hierarchical Indexing) and a 135B N-gram embedding, trained on 35T+ tokens.
Four Chinese open-weight MoE models — deepseek-ai/DeepSeek-V4-Pro-DSpark (last modified 2026-07-08), zai-org/GLM-5.2 (last modified 2026-07-02), tencent/Hy3 (last modified 2026-07-06), and meituan-longcat/LongCat-2.0 (last modified 2026-07-08) — were all visible on the Hugging Face trending list by 2026-07-08; each model's own card documents a sparse-attention design at long context (DeepSeek-V4-Pro CSA+HCA Hybrid Attention at 1M context; GLM-5.2 Glm MoE DSA with IndexShare at 1M context claiming 2.9x per-token FLOPs reduction; Tencent Hy3 MTP layer at 256K context; Meituan LongCat-2.0 LongCat Sparse Attention at undisclosed context with a 135B N-gram embedding and 35T+ training tokens).

Evidence pipeline

From the news

Breakdown

Reading DeepSeek-V4-Pro / GLM-5.2 / Tencent Hy3 / Meituan LongCat-2.0 individually turns into benchmark reading. This piece explains how to use the 'sparse attention at long context is the strongest cross-cutting research-side signal of early July' frame — group the four models by architectural similarity (DeepSeek CSA+HCA / GLM IndexShare / Hy3 MTP / LongCat LSA) so creators can produce 'the four Chinese open-weight MoEs landed on HF in one week and they're all betting on sparse attention at long context' content.

Sources

Risks

Pin links to each source; quote only what the captured summary states; do not paraphrase specific benchmark numbers, performance metrics, paper claims, or architectural details beyond what is stated.
Pin the link to each model card; quote only what the card states; do not paraphrase self-reported benchmark numbers as third-party validation; flag that all four numbers are self-reported on the model cards.

Demo ideas

Load DeepSeek-V4-Pro, GLM-5.2, Hy3, and LongCat-2.0 from Hugging Face and run the same 128K-context long-document inference to show each model's sparse-attention behavior.
Build a 'Chinese July MoE wave' architecture + self-reported-benchmark comparison card, flagging which numbers haven't been independently reproduced.
Record a 'DeepSeek-V4-Pro + DSpark speculative decoding' long-context generation demo to show the semi-autoregressive generation speed delta.