grok catch still emits metrics errors

[ryanpodonnell1]

Hello,

I can't seem to figure out why i'm getting metrics errors (processor_failed) to CloudWatch even though it should be cleared by the catch block:

          processors:
          - label: extract_squid_messages
            grok:
              pattern_definitions:
                "SQUID_PROXY_EGRESS": '(?P<RequestTimeStamp>\d+\.\d+)\s+(?P<NetworkDuration>\d+)\s(?P<SrcIpAddr>(?:[0-9]{1,3}\.){3}[0-9]{1,3})\s+(?P<Action>\w+\/[0-9]{1,3})\s+(?P<RequestSizeInBytes>\d+)\s+(?P<Method>\w+)\s+(?P<URI>[^\s]+)\s+(?P<Username>[^\s]+)\s+(?P<HierarchyRoute>\w+\/[^\s]+)\s+(?P<ContentType>(-|\w+\/\w+))'
              expressions:
                - '%{SQUID_PROXY_EGRESS}'
          - catch:
            - label: catch_grok_errors #if a pattern doesn't match the grok pattern it will error, just clearing the error so we can drop in the main pipeline later
              mutation: |
                #!blobl
                  meta "drop" = "true"

output:

DEBU No matches found for payload: 2024-03-29 00:02:33.288996 File downloaded to /tmp/passlist.txt  @service=benthos label=extract_squid_messages path=root.branch.processors.0
DEBU Processor failed: no pattern matches found    @service=benthos label=extract_squid_messages path=root.branch.processors.0
DEBU No matches found for payload: 1711670554.447 RELEASE -1 FFFFFFFF 7F180000000000000800000000000000  200 1711670554 1711670554 1711670554 text/plain;charset=utf-8 -1/1235 GET cache_object://localhost/counters  @service=benthos label=extract_squid_messages path=root.branch.processors.0
DEBU Processor failed: no pattern matches found    @service=benthos label=extract_squid_messages path=root.branch.processors.0

Basically I want to pattern match a grok pattern and drop everything else. Is there a more optimal way to do this?

[mihaitodor]

Hey @ryanpodonnell1 👋 I'm guessing you're referring to the processor_error metric which is emitted for every processor in the pipeline. It's not possible to avoid having it reported for the extract_squid_messages processor when it throws an error even if you use catch after it to clear that error from the message. You can replace it with a mapping processor where you can call the relevant bloblang regex methods to achieve the same thing if those metrics are giving you headaches, but I think it's easier to filter out processors you don't want to monitor in CloudWatch based on labels, since you'll likely have more situations like this one as your pipelines become more complex in the future.

[ryanpodonnell1]

sorry for the late reply -- I'll give this a whirl thanks!