[Other]: Missing Blog Posts in Production Builds on Cloudflare Pages

[KardeniaPoyu]

Issue Description

This problem has occurred multiple times during my deployments, and I read all of the issues but found nothing.

Some blog posts fail to render in production (Cloudflare Pages) while working correctly in local development. This results in missing HTML files (e.g., dist/posts/<ID>/index.html) without explicit build errors during the Astro phase.

I initially suspected formatting issues (especially frontmatter and Markdown structure), and spent considerable time debugging and validating content consistency. This included repeatedly checking for syntax errors and eliminating potential parsing issues.

After extensive testing, the problem still persisted intermittently, even with clean and validated content.

Based on these observations, it seems more likely that the issue is related to resource constraints (e.g., memory limits in the Cloudflare Pages build environment), rather than purely content formatting errors.

The issue is strongly correlated with large local images and complex LaTeX content, leading to "all-or-nothing" failures where existing posts also disappear due to the failed deployment.

It might also be related to Astro's build process under constrained environments, which means Fuwari has nothing to do with this problem.

But I’d also like to share the workarounds I’ve identified so far, and would appreciate any feedback or better approaches to address this issue.

---

Environment

• Deployment Platform: Cloudflare Pages
• Node Version: v22.16.0
• Astro Version: v5.13.10
• Fuwari Version: commit 341b6ba
• Package Manager: pnpm@9.14.4
• Cloudflare configuration:

Build command: pnpm build
Build output: dist
Root directory:  `-`
Build comments: Enabled
Build cache: Disabled
Build system version: Version 3

---

Reproduction

1. Asset Load: Add multiple posts containing high-resolution images (>1.5MB) in src/content/posts.
2. Concurrent Push: Commit and push a batch of these posts simultaneously to GitHub.
3. Deployment: Observe the Cloudflare Pages build log. The heavy resource demand (Sharp image processing + LaTeX) often exceeds the build environment's limits, causing silent skips.
4. Verification: Pushing these same posts one by one (individual commits/deployments) or removing large assets often bypasses the issue, confirming a resource/concurrency bottleneck.

---

Possible Causes

1. Resource Constraints (OOM / Timeout)

Cloudflare Pages' build workers have strict memory limits. Astro's default high-concurrency build for image processing (via Sharp) and LaTeX rendering can exhaust available RAM, leading to incomplete output.

2. LaTeX / Unicode Parsing

Unrecognized Unicode characters in math mode (e.g., ⓒ, ⓓ) or un-wrapped Chinese and Japanese characters can trigger KaTeX warnings/errors that may stall the rendering process for specific routes.

---

Build Integrity Audit

For those who has the same problem, it is recommended to add a post-build audit script.

This is because Astro (and thus Fuwari) will often complete the build successfully even if some pages were dropped due to worker failures. The resulting site will appear to function normally, but specific posts will be missing from the production build. This audit script acts as a safeguard to ensure every non-draft post is correctly generated before deployment.

1. Create scripts/check-dist.js

import fs from 'fs';
import path from 'path';

// Simplified file-system based audit for CI environment
const POSTS_DIR = './src/content/posts';
const DIST_DIR = './dist/posts';

const sourcePosts = fs.readdirSync(POSTS_DIR).filter(f => 
  fs.statSync(path.join(POSTS_DIR, f)).isDirectory()
);

const missing = [];
sourcePosts.forEach(id => {
  const mdPath = path.join(POSTS_DIR, id, 'index.md');
  if (fs.existsSync(mdPath)) {
    const content = fs.readFileSync(mdPath, 'utf-8');
    // Robust check for draft status in frontmatter
    const isDraft = /draft:\s*true/.test(content);
    const htmlPath = path.join(DIST_DIR, id, 'index.html');
    
    if (!isDraft && !fs.existsSync(htmlPath)) {
      missing.push(id);
    }
  }
});

if (missing.length > 0) {
  console.error(`\x1b[31m[ERROR] Post Rendering Audit Failed!\x1b[0m`);
  console.error(`Missing ${missing.length} posts in production build:`);
  missing.forEach(id => console.error(` - ${id} (Expected: ${path.join(DIST_DIR, id, 'index.html')})`));
  process.exit(1); // Force CI failure
} else {
  console.log('\x1b[32m[SUCCESS] All posts verified in dist/.\x1b[0m');
}

2. Integrate into package.json

"scripts": {
 "build": "astro build && pagefind --site dist && node scripts/check-dist.js"
}

---

Workarounds & Solutions

1. Incremental Deployment (Confirmed Fix)

Instead of batch uploading, push posts individually. This reduces the peak memory load on the build server at any single time. Delete the cover.png for some posts.

2. Limit Build Concurrency

Force Astro to use fewer workers to stay within memory limits:

"build": "astro build --concurrency 1 && pagefind --site dist"

3. Optimize Assets (Pre-processing)

Manually compress images (target < 500KB) before committing. For very large decorative images, moving them to the public/ folder bypasses the expensive processing engine.

4. Schema Robustness

Use z.coerce or .catch() in src/content/config.ts to ensure that a single malformed post doesn't break the entire collection.

5. Move assets to public/:

I am aware that moving images to the public/ folder and using absolute paths can bypass Astro's build-time image processing (Sharp), thus avoiding the memory overhead. However, for a better co-location of content and assets, many users (including myself) prefer keeping images within the content collection folders. The above concurrency workaround allows maintaining this structure while staying within resource limits.

---