-
Notifications
You must be signed in to change notification settings - Fork 209
FAQ on what happens when i use a coding agent #2799
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
annabellscha
wants to merge
2
commits into
main
Choose a base branch
from
add-FAQ-on-coding-agents
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,58 @@ | ||
| --- | ||
| title: What happens when I use Langfuse's agentic onboarding with a coding agent? | ||
| description: Learn what Langfuse's coding-agent onboarding does automatically for tracing and prompt management. | ||
| tags: [agent-skill, observability-get-started, prompt-management-get-started] | ||
| --- | ||
|
|
||
| # What happens when I use Langfuse's agentic onboarding with a coding agent? | ||
|
|
||
| It is not just pasting boilerplate into your repo. | ||
|
|
||
| When you use Langfuse's agentic onboarding with the [Langfuse Agent Skill](/docs/api-and-data-platform/features/agent-skill) or the Cursor plugin that bundles it, the coding agent first inspects your codebase and then chooses the Langfuse workflow that fits what it found. | ||
|
|
||
| ## For tracing | ||
|
|
||
| If you ask the agent to instrument your app, it will typically: | ||
|
|
||
| - Detect which LLM libraries or frameworks you already use, such as OpenAI, Anthropic, LangChain, or other supported integrations | ||
| - Prefer the matching Langfuse integration instead of falling back to generic manual instrumentation | ||
| - Add the required imports, environment-variable usage, and setup code | ||
| - Refactor existing LLM calls so they are traced with the right integration pattern | ||
| - Keep the integration close to your current code instead of rewriting the whole flow | ||
|
|
||
| Typical prompt: | ||
|
|
||
| ```txt | ||
| Add tracing to this application with Langfuse following best practices. | ||
| ``` | ||
|
|
||
| ## For prompt management | ||
|
|
||
| If you ask the agent to work with prompts, it can: | ||
|
|
||
| - Create a prompt in Langfuse with the variables you describe | ||
| - Migrate hardcoded prompts from your codebase into Langfuse | ||
| - Replace inline prompt strings with fetched prompts in application code | ||
| - Fetch a specific prompt version or label, such as `production`, and wire it into the relevant code path | ||
|
|
||
| Typical prompts: | ||
|
|
||
| ```txt | ||
| Create a prompt in Langfuse called "movie-critic" that takes a movie name and critic level as variables. | ||
| ``` | ||
|
|
||
| ```txt | ||
| Migrate the hardcoded prompts in this codebase to Langfuse. | ||
| ``` | ||
|
|
||
| ```txt | ||
| Fetch the "movie-critic" prompt, production version, from Langfuse and use it in my application. | ||
| ``` | ||
|
|
||
| ## Why it works better than a generic coding prompt | ||
|
|
||
| The skill gives the agent Langfuse-specific instructions for instrumentation, prompt migration, documentation lookup, and common workflows. That means the agent is not guessing how Langfuse should be used in your stack. It has explicit guidance on what to look for and which implementation patterns to prefer. | ||
|
|
||
| ## What you should expect | ||
|
|
||
| In a good run, the output is a focused diff that wires Langfuse into the code you already have. You should still review the changes before merging. The agent can automate most of the setup, but the final result still depends on your environment variables, your application structure, and whether the agent can clearly infer the right integration points from the repo. | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🟡 The new FAQ's tags
observability-get-startedandprompt-management-get-startedwill surface this informational how-it-works article inside the "Not seeing what you expected?" troubleshooting sections of the observability and prompt-management get-started pages, which is a content mismatch. Consider removing those two tags if the FAQ is only intended for the agent-skill page. Note: the concern about a 404 on the agent-skill tag page is not valid —page.tsxhas a dynamic fallback (lines 43-58) andgenerateStaticParams(lines 222-233) auto-generates routes for any tag found in FAQ frontmatter, so/faq/tag/agent-skillwill resolve correctly.Extended reasoning...
Content placement mismatch via get-started tags
The new FAQ file
agentic-onboarding-with-coding-agents.mdxcarries three tags:agent-skill,observability-get-started, andprompt-management-get-started. TheFaqPreviewcomponent (components/faq/FaqPreview.tsx:19) filters FAQ pages by checking if any of a page's tags match the requested tag. Becauseobservability/get-started.mdx(line 425) calls<FaqPreview tags={["observability-get-started"]} />andprompt-management/get-started.mdx(line 127) calls<FaqPreview tags={["prompt-management-get-started"]} />, the new article will appear in both of those pages' "Not seeing what you expected?" sections.Those sections exist as troubleshooting helpers — their current FAQs are things like "I have setup Langfuse but do not see traces", "empty trace input/output", "old prompt version caching". The new article is an educational how-it-works piece explaining what the coding-agent onboarding does automatically. It answers a different class of question and is not troubleshooting content. Surfacing it there could confuse users who are actively looking for help with missing data or incorrect behaviour.
Concrete proof: Navigate to
/docs/observability/get-started, scroll to "Not seeing what you expected?". After this PR, that section will list the new article alongside "I have setup Langfuse but do not see traces" — an obvious editorial mismatch in intent and audience.Refutation of the 404 concern: Several verifiers flagged that
/faq/tag/agent-skillwould 404 because nocontent/faq/tag/agent-skill.mdxfile exists. This is not correct.app/[section]/[[...slug]]/page.tsxlines 43-58 implement a special case: when no MDX page is found for a/faq/tag/[tag]URL, it renders a dynamicFaqPreviewfiltered to that tag. Furthermore,generateStaticParams()(lines 222-233) iterates all FAQ pages, collects every tag found in frontmatter, and adds static params for any tag that lacks a dedicated content file. So/faq/tag/agent-skillwill be statically generated and will render correctly — no 404 will occur. The "View all" link inFaqIndexis safe.Why the content issue is real despite intentionality: One refuter argued the tags were deliberately chosen. That may be so, but deliberate tag selection that places an educational article inside a troubleshooting section is still an editorial mistake. Intention does not equal correct placement. The observability and prompt-management get-started troubleshooting sections serve a specific audience (users who have set up the integration but are not seeing data), and inserting a "how does agentic onboarding work" article there does not serve that audience.
Fix: Remove
observability-get-startedandprompt-management-get-startedfrom the new file's tags list, leaving onlyagent-skill. If broader visibility of the agentic onboarding flow is desired across those pages, consider a dedicated informational section rather than piggybacking on the troubleshooting FAQ component.