Topics scaffoldings by fabioscagliola · Pull Request #70 · Zuehlke/archetypes

fabioscagliola · 2025-12-02T18:25:25Z

I added Christian's fork containing the AI scaffoldings as remote.
I created a branch from his main.
I manually performed some cleanup:

I removed redundant headings as "learning resources" is added automatically
I removed redundant descriptions from bodies as they now come from the frontmatter

Finally, I wrote a dead links killer script and ran it against all the topics. Dead links are now dead for good!
This changed the frontmatter of all the topics in terms of YAML, formally but not substantially.
I browsed through all the topics: they all look good!

Copilot

Pull request overview

This pull request adds AI-generated scaffolding content to the learning resources project, implementing a systematic approach to populate missing or stub topic pages with structured content and learning resources. The PR includes a Python script to remove dead links and performs YAML frontmatter reformatting across all topic files.

Key Changes

Dead link removal script: Adds kill_the_dead_links.py to validate and remove inaccessible URLs from learning resources
AI scaffolding documentation: Updates README.md with guidelines for AI-generated content including marking, structure, and resource selection criteria
YAML formatting standardization: Reformats all topic frontmatter from inline flow style to block style (e.g., - type: external_link instead of - type: "external_link")

Reviewed changes

Copilot reviewed 80 out of 82 changed files in this pull request and generated 18 comments.

Show a summary per file

File	Description
scripts/kill_the_dead_links.py	New Python script to validate URLs and remove dead links from topic learning resources
scripts/requirements.txt	Dependencies for the dead links script (python-frontmatter, requests)
README.md	Documents AI scaffolding approach, content guidelines, and resource selection criteria
.github/copilot-instructions.md	Adds references to design principles and data architecture documentation
docs/design-principles.md	Formatting improvements with consistent blank lines
docs/data-architecture.md	Formatting improvements with consistent section spacing
src/topics/*.md (87 files)	YAML frontmatter reformatting and addition of AI-generated content with descriptions, learning resources, and cross-references

Comments suppressed due to low confidence (1)

scripts/kill_the_dead_links.py:23

Except block directly handles BaseException.

            except:

Copilot · 2025-12-02T18:36:59Z

scripts/kill_the_dead_links.py

+def kill_the_dead_links(path):
+    print(f"Processing file \"{path}\"..")
+    post = frontmatter.load(path)
+    if "learning_resources" in post.metadata:
+        for resource in post.metadata["learning_resources"]:
+            print(f"Requesting URL \"{resource['url']}\"..")
+            try:
+                response = requests.get(resource["url"], timeout=5)
+                response.raise_for_status()
+            except:
+                print(f"Dead link found: {resource['url']}")
+                post.metadata["learning_resources"].remove(resource)


Modifying a list while iterating over it with remove() can cause items to be skipped. When an item is removed, the iterator's index doesn't adjust, potentially skipping the next element. Use a list comprehension or iterate over a copy: post.metadata["learning_resources"] = [resource for resource in post.metadata["learning_resources"] if not is_dead(resource)]

Suggested change

def kill_the_dead_links(path):

print(f"Processing file \"{path}\"..")

post = frontmatter.load(path)

if "learning_resources" in post.metadata:

for resource in post.metadata["learning_resources"]:

print(f"Requesting URL \"{resource['url']}\"..")

try:

response = requests.get(resource["url"], timeout=5)

response.raise_for_status()

except:

print(f"Dead link found: {resource['url']}")

post.metadata["learning_resources"].remove(resource)

def is_dead(resource):

print(f"Requesting URL \"{resource['url']}\"..")

try:

response = requests.get(resource["url"], timeout=5)

response.raise_for_status()

return False

except:

print(f"Dead link found: {resource['url']}")

return True

def kill_the_dead_links(path):

print(f"Processing file \"{path}\"..")

post = frontmatter.load(path)

if "learning_resources" in post.metadata:

post.metadata["learning_resources"] = [

resource for resource in post.metadata["learning_resources"]

if not is_dead(resource)

]

Copilot · 2025-12-02T18:36:59Z

README.md

+  - Learning resources need to come from sources widely accepted in expert communities
+  - Original documentation is preferrable
+  - Youtube videos from widely accepted channels with a long history of reliable information are acceptable
+  - Freely accessible ressources are preferred to paid-for offers


Typo: "ressources" should be "resources".

Suggested change

- Freely accessible ressources are preferred to paid-for offers

- Freely accessible resources are preferred to paid-for offers

Copilot · 2025-12-02T18:37:00Z

README.md

+- Provide the structure as laid out in the data architecture
+- Add learning resources for all levels
+  - Learning resources need to come from sources widely accepted in expert communities
+  - Original documentation is preferrable


Typo: "preferrable" should be "preferable".

Suggested change

- Original documentation is preferrable

- Original documentation is preferable

Copilot · 2025-12-02T18:37:00Z

README.md

+- Add learning resources for all levels
+  - Learning resources need to come from sources widely accepted in expert communities
+  - Original documentation is preferrable
+  - Youtube videos from widely accepted channels with a long history of reliable information are acceptable


Capitalization: "Youtube" should be "YouTube" (proper brand name).

Suggested change

- Youtube videos from widely accepted channels with a long history of reliable information are acceptable

- YouTube videos from widely accepted channels with a long history of reliable information are acceptable

Copilot · 2025-12-02T18:37:00Z

README.md

 -  `src/` - This folder contains definition files that outline the archetypes, topics, and learning resources in Markdown format.
    -  `src/archetypes/` - Contains archetype definitions, which describe the competencies and skills expected at various stages of a software engineer's career.
-    -  `src/topics/` - Contains definitions of topics that are relevant to the archetypes, detailing the specific skills, knowledge areas and any learning resources associated with them.
+    -  `src/topics/` - Contains definitions of topics that are relevant to the archetypes, detailing the specific skills, knowledge areas and any learning resources associated with them. 


Trailing whitespace on this line. Consider removing it to maintain consistency.

Suggested change

- `src/topics/` - Contains definitions of topics that are relevant to the archetypes, detailing the specific skills, knowledge areas and any learning resources associated with them.

- `src/topics/` - Contains definitions of topics that are relevant to the archetypes, detailing the specific skills, knowledge areas and any learning resources associated with them.

Copilot · 2025-12-02T18:37:03Z

src/topics/user-stories.md

+{{ render_description() }}

- 
+{{ render_learning_resources() }}


{{ render_learning_resources() }} outputs link title and url from frontmatter without validation/escaping, allowing injection of malicious links (e.g., javascript: URLs) or HTML in titles. An attacker could submit a resource that executes JS when clicked. Fix by strictly validating URLs to http/https schemes and escaping titles/URLs in the macro before rendering.

Copilot · 2025-12-02T18:37:04Z

src/topics/threat-modelling.md

+*Generated by AI, pending community review.*

-## Learning Materials
+{{ render_description() }}


{{ render_description() }} returns frontmatter description directly without sanitization, enabling stored XSS if HTML/JS is included in the description. A malicious PR could add <script> in frontmatter that executes when the page is viewed. Fix by escaping/sanitizing in render_description() and/or enabling HTML sanitization in MkDocs Material.

Copilot · 2025-12-02T18:37:04Z

src/topics/threat-modelling.md

+{{ render_description() }}

- 
+{{ render_learning_resources() }}


{{ render_learning_resources() }} emits titles/URLs from frontmatter without escaping or URL scheme checks, allowing injection of javascript: links or HTML in titles. A crafted resource could cause script execution on click. Fix by whitelisting http/https schemes and escaping titles/URLs inside render_learning_resources().

Copilot · 2025-12-02T18:37:04Z

src/topics/technical-documentation-writing.md

+*Generated by AI, pending community review.*

-## Learning Materials
+{{ render_description() }}


The newly added {{ render_description() }} macro outputs raw frontmatter without sanitization, creating a stored XSS risk if HTML/JS is placed in description. An attacker could add <script> in a PR and have it execute for readers. Fix by escaping/sanitizing in render_description() and configure MkDocs to disallow raw HTML or only allow a safe subset.

Copilot · 2025-12-02T18:37:04Z

src/topics/technical-documentation-writing.md

+{{ render_description() }}

- 
+{{ render_learning_resources() }}


{{ render_learning_resources() }} renders title and url directly from frontmatter, allowing injection of unsafe links (e.g., javascript:) or HTML in titles. A malicious resource could trigger script execution on click. Fix by validating URL schemes (only http/https) and escaping titles/URLs inside the macro.

pburls

Something doesn't look to have worked correctly. Some working links from the FSI topics were removed.

pburls · 2025-12-10T08:02:45Z

src/topics/accounting-basics.md

 learning_resources:
-  - type: "external_link"
-    title: "Accounting Equation: What It Is and How You Calculate It"
-    url: "https://www.investopedia.com/terms/a/accounting-equation.asp"


issue: I checked two of the investopedia links that were removed and they both worked. Is it possible that the script incorrectly removed them due to some transient issue?

Christian Heger and others added 9 commits December 1, 2025 13:53

Update file content-prompt.md

807cb2c

Instructions for AI scaffolding

5d3d0a0

Specified AI scaffolding

d9f099d

Add AI scaffolded Topic pages

66c1768

Remove redundant title

de5b12a

Remove chomping indicators

3b26c15

Removed descriptions from body

39f450a

Dead links killer script

a32ec22

Dead links killed for good

a5a0227

fabioscagliola requested a review from Copilot December 2, 2025 18:26

Copilot started reviewing on behalf of fabioscagliola December 2, 2025 18:26 View session

fabioscagliola requested a review from pburls December 2, 2025 18:27

Copilot finished reviewing on behalf of fabioscagliola December 2, 2025 18:31

Copilot AI reviewed Dec 2, 2025

View reviewed changes

Restoring (cherry-picking) files not relevant to this PR

43da665

fabioscagliola changed the title ~~Scaffoldings~~ Topics scaffoldings Dec 3, 2025

pburls suggested changes Dec 10, 2025

View reviewed changes

	- Freely accessible ressources are preferred to paid-for offers
	- Freely accessible resources are preferred to paid-for offers

	- Original documentation is preferrable
	- Original documentation is preferable

	- Youtube videos from widely accepted channels with a long history of reliable information are acceptable
	- YouTube videos from widely accepted channels with a long history of reliable information are acceptable

	- `src/topics/` - Contains definitions of topics that are relevant to the archetypes, detailing the specific skills, knowledge areas and any learning resources associated with them.
	- `src/topics/` - Contains definitions of topics that are relevant to the archetypes, detailing the specific skills, knowledge areas and any learning resources associated with them.

Conversation

fabioscagliola commented Dec 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Key Changes

Reviewed changes

Uh oh!

Copilot AI Dec 2, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 2, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 2, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 2, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 2, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 2, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 2, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 2, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 2, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 2, 2025

Choose a reason for hiding this comment

Uh oh!

pburls left a comment

Choose a reason for hiding this comment

Uh oh!

pburls Dec 10, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

fabioscagliola commented Dec 2, 2025 •

edited

Loading