Aggregation of multiple CAN messages in 1722 frame

[adriaan-niess]

Based on comment from @JLPLabs we should consider adding support to aggregate multiple CAN frames into a single Ethernet frame.

Adriaan,
You'll see a 3rd commit on this PR.

NOTE PLEASE LET ME KNOW:
a) if this is more than you want,
b) if there is a better / more idiomatic approach I should be taking.

This last change can be summarized as:

• LISTENER

  • updated one line in acf-can-listener so that the acf_pdu pointer is moved through the ethernet frame. (right now there is logic that seems to imply that was the desired behavior, but the pointer wasn't actually being updated).

• TALKER

  • added a --count COUNT flag to the command line. This allows the user to require COUNT number of CAN frames to be packed into one Ethernet frame.
  • used the variable that already existed, num_acf_msgs for handling the count logic.

Originally posted by @JLPLabs in #18 (comment)

[adriaan-niess]

@JLPLabs, At first glance I think it would be a little bit more flexible to provide a timeout (longest possible time allowed to buffer a CAN message) to the talker application instead of a fixed number of CAN messages to be aggregated. Then the talker would either wait until

• Timeout for oldest CAN message in the send buffer elapses
• Or buffered CAN messages (plus headers) exceed the size of the Ethernet MTU (max transmission unit)

For the default setting we could set the timeout simply to zero which would imply no buffering at all.

But maybe you had a very specific reason why you did it the way you did?

[JLPLabs]

Adriaan,

I like those ideas! And thank you for asking if I had a specific reason. I do, but it is perhaps a little selfish...

I do like the idea of a max number of CAN frames per Ethernet frame, as there may be use cases where the user (particularly while doing initial development, which is me right now!) prefers looking at only a handful of CAN frames in an Ethernet frame while analyzing protocol behavior.

I'm going to take the liberty of combining your idea with mine... I hope you are OK with that.

Let's say the default behavior of talker is "full buffering"-- unless the user indicates otherwise, we'll pack as many CAN frames into the Ethernet frame as we can before sending it.

If the user doesn't want "full buffering" then they can use one or both of --timeout and/or --count.

Here are behaviors other than default:

1. --count 1 -- today's behavior of talker (we send the Ethernet frame as soon as we get CAN frame).

2. --timeout 0 -- also gives today's behavior.

3. --count 5 -- do some packing, but keep it user-friendly when doing manual inspection of frames

4. --timeout 400 -- pack only "bursty" traffic

Formal Definitions

• --timeout <time> where <time> is integer number of microseconds (not milliseconds), so that it is possible to support the use case where a user only wants packing done for "bursts" of traffic.

  • minimum value: 0
  • maximum value: (2^32 - 1) usec [1]
  • default: maximum value

  examples:

  • 0 means just one CAN frame per 1722 frame.
  • 400 means we are done waiting for CAN frames if more than 400 usec has passed since the last CAN frame was loaded into the buffer.

• --count <count> where <count> is the maximum number of CAN frames allowed in an Ethernet frame.

  • minimum: 1
  • maximum: (2^32-1) (which really means, "when the Ethernet frame is full". The actual count for a full Ethernet frames depends on Ethernet MTU and size of CAN/CAN FD(/CAN XL?) frames)
  • default: maximum value

Interaction

The Ethernet frame is sent once any of these are true:

• the time since the last CAN frame was packed is > <time> usec.
• number of CAN messages packed == <count>
• the next CAN frame to pack will overflow the Ethernet MTU (so send the Ethernet frame and start a new one)

--notes--
[1] This "only" gives the user 1.19 hours between messages... which for nearly all practical vehicle use is OK. (I could imagine some diagnostics done by a product engineer -- where they filter traffic looking for a very specific and rare event and they want all of theses rare events in one Ethernet frame and 1.19 hours isn't enough). For this case the engineer should create their own application?? Otherwise the tradeoff could be to measure <time> in ms, and which means we can no longer require packing to be done for just "bursts" of traffic.

[adriaan-niess]

Hey, this looks very well thought out to me. Big thanks! Unfortunately I can only give you a short reply, because I'm in a little bit of hurry. Some thoughts:

• Seems plausible to me now that you might want to limit the number of CAN frames for debugging purposes, let's keep that 👍
• I think it makes more sense to interpret the timeout with respect to the oldest (not the newest) message in the buffer. The reason is that you want to be able to guarantee a deadline (or worst-case transmission time) for your CAN messages. For example you might want to ensure that certain sensor signals are delivered to your central compute in under <100ms. If you reset the timeout every time a new CAN message is added to the buffer, it becomes quite hard to estimate for how long CAN frames could get delayed in the worst-case.
• Having ~1h as the max timeout seems more than enough to me. I don't think there are use cases where it's acceptable to wait 1h to receive the data even it's not that important. I'd estimate that in most real-world scenarios the timeout would be set to <1s.

[JLPLabs]

Sorry for my delayed response.
I think your logic for timing makes sense. I'll start to work on updated code with this definition for --timeout:

• --timeout <time> where <time> is an integer number of microseconds, "usec". (not milliseconds)

  • The timeout establishes the maximum wait for retransmission of a CAN message.
  • The timeout clock starts with the arrival of the first message.
  • The retransmission may be sooner than <time> usec if the Ethernet frame becomes full or the maximum count (see --count) of CAN messages has arrived.
  • minimum value: 0
  • maximum value: (2^32 - 1)
  • default: maximum value

  examples:

  • --timeout 0 means no waiting, immediately pack and send the CAN frame in a 1722 frame.
  • --timeout 700 means wait no longer than 700 usec from the arrival of the first CAN frame to start the 1722 frame transmission process.
  • if no value for --timeout is given, then the 1722 frame will be launched when one of these is true:
    • the 1722 frame is full
    • the 1722 frame has <count> number of messages
    • 1.193 hours has passed since the arrival of the first message (in a real network this effectively never going to happen)

[adriaan-niess]

Hi,

don't worry take your time. Only thing I'd change is to set the default timeout to 0 instead of 2^32-1 because then it could happen that a single CAN frame gets delayed for a very long time. I fear some users could interpret this as a bug if they don't study the documentation first.

If you need any help or feel stuck at some point let me know.

Regards
Adriaan