Searching for trace span events

[m3nowak]

Is your question related to a specific component?

LogsQL

Describe the question in detail

Question

I'd like to search span events by their event name in VictoriaTraces using LogsQL, and I'm not sure what the recommended approach is.

Context

From what I understand, OTLP span events are stored as indexed fields inside the span, for example:

• event:0:event_time_unix_nano
• event:0:event_name
• event:0:event_attr:<attr>
• event:1:event_time_unix_nano
• event:1:event_name
• event:!:event_attr:<attr>

where the number is the event index within the span.

Problem

I would like to search for spans containing an event with a given name, for example:

• exception
• db.retry
• payment.failed

The difficulty is that the field name is not stable, because the event index can be different for every span:

• event:0:event_name
• event:1:event_name
• event:2:event_name
• ...

So I don't know how to build a LogsQL query that matches event names regardless of their index.

What I'm asking

Is there currently a recommended way to search span events by event name in VictoriaTraces?

For example, is there some LogsQL pattern, pipe, or field-selection approach that can be used to search across fields like:

• event:0:event_name
• event:1:event_name
• event:N:event_name

without knowing the event number in advance?

Any guidance or examples would be appreciated.

Key concepts docs

• VictoriaTraces key concepts - https://docs.victoriametrics.com/victoriatraces/keyconcepts/
• LogsQL - https://docs.victoriametrics.com/victorialogs/logsql/

[jiekun]

Hello. We're aware of the issue because LogsQL doesnt support search across multiple fields now. What's under development is the regex (or prefix suffix) filter on field name. For example, the issue could be solved by LogsQL ''event:event_name:.*=something''.

We need to wait until this feature became available in VictoriaLogs and LogsQL first.

[m3nowak]

Guess I'll have to wait until VictoriaMetrics/VictoriaLogs#82 is solved

[jiekun]

Yes. We have an ongoing effort on that feature, so once it's released, we'll port to VictoriaTraces because the Tempo API also needs this feature.

[jiekun]

Hi, just FYI VictoriaMetrics/VictoriaLogs#82 has been implemented.
We’ll need to update the logstorage dependency to enable this support in VictoriaTraces.