Skip to content

Add headings and other geometric pins in motion using State_tags#3995

Open
rodw-au wants to merge 38 commits into
LinuxCNC:masterfrom
rodw-au:state-tags-arcs
Open

Add headings and other geometric pins in motion using State_tags#3995
rodw-au wants to merge 38 commits into
LinuxCNC:masterfrom
rodw-au:state-tags-arcs

Conversation

@rodw-au
Copy link
Copy Markdown
Contributor

@rodw-au rodw-au commented May 2, 2026

Outline

create new [motion.interp.xx ] pins for heading and various geometric data (refer man motion for this PR). Note the following use cases behind this PR.

  • heading requested by Andy as it was trivial to add while working on this. Its been in Mach3 for years. Useful for tangential knives and saws.
  • normal-heading angle from current position to arc centre. Allow bevelled cutting in conjunction with heading. This could allow cuttings of bevels (eg with a plasma torch) with 2D gcode. While torch holders exist that keep the angled torch tip axial to the Z axis, external offsets could be used to calculate the desired XYZ coordinates to centre the torch tip
  • radius A theory of mine. Plasma cutters use torch voltage as a process contol variable to set torch height to obtain even kerf but this assumes a constant velocity (reducing velocity increases torch voltage). When the torch aproaches a centrepetal radius limit, the system could lock the height and switch to using current as the process control variable to keep the voltage constant.
  • iscircle allows a plasma cutter to detect bolt holes (where arc start position = end position) and apply custom hole processing rules more inteligently than the current state. Can be used in conjunction with normal-heading for countersinking holes.

I am sure the community will find many other use cases.

Code notes

  • Uses an extension of state_tags.
  • New procedures tag_straight() and tag_arc() in interp_convert.cc apply our new tags for straight moves and arcs.
  • Because these tags are only applied at the beginning of a segment, heading calculations are applied in motion/control.c/update_status() in real time.
  • The rest of the code is mechanical to transfer the tags added by the interpreter back to motion so the data can be used in real time.
  • May provide a framework for using additional tags in motion

Caveats

We use the state tag motion_type (GM_FIELD_MOTION_MODE) to detect G0,G1,G2,G3 in motion. Currently there is no way to use existing interpreter equates for these values so the numbers are hard coded as 0,10,20,30.

RunTests

These pass locally

Runtest: 278 tests run, 278 successful, 0 failed + 0 expected, 4 skipped, 0 shmem errors
rod@debian:~/devt/linuxcnc$ 

Acknowledgements

Many thanks to Luca Toniolo (grandixximo) for his encouragement and help resolving errors with the tests.

Rod and others added 14 commits April 13, 2026 17:44
merge latest upstream into state-tags-arcs branch
merge master to state-tag-arcs branch
…d read

active_modes() in rs274ngc_pre.cc still ran the original unclamped loop
above the clamped one, so the stack overflow into saved_settings[5] was
unchanged. The line-number copy was also commented out. Drop the duplicate
and restore the line-number copy.

write_state_tag() still dereferenced block on the iscircle assignment
(`block->iscircle = (block == NULL) ? 0 : block->iscircle`), reintroducing
the M70 NULL-deref. Replace with a plain null-guarded read.

control.c output_to_hal() read the iscircle bit out of `tag.fields[]` using
the StateFlag index GM_FLAG_IS_CIRCLE, which indexes past `fields[]`
(GM_FIELD_MAX_FIELDS == 8). Read it from packed_flags instead, like the
update_status() copy does. Also dropped the duplicate
interp_normal_heading write in the G2/G3 branch and restored the
WATCH_FLAGS debug block to live inside update_status().
Round 2: re-fix M70 segfault and iscircle handling
M_PI_2 is a glibc extension and is not exposed by RTAI's rtapi_math.h,
so the RTAI build fails with 'M_PI_2 undeclared' at control.c:2283.
M_PI is available in both, so use M_PI/2.0 instead.
@BsAtHome
Copy link
Copy Markdown
Contributor

BsAtHome commented May 3, 2026

Some observations:

  • Why is there a file called arc_heading?
  • I'm not sure that creating test-files in the root LCNC directory called nc_files is a good idea. Tests are supposed to be in the tests directory and have a real test associated. If they are not for testing, then you need to have a specific config and the should be created in the config/sim/... somewhere as an example.
  • The line docs/man/man9/output_buffer.9 is malplaced in the root .gitignore and should be in the docs/man/.gitignore file. However, it is probably malplaced in this PR as it seems unrelated to this PR.
  • The diff of src/emc/rs274ngc/interp_convert.cc is 7000+ lines. Most, if not all, seems to be inappropriate (or stylistic wrong) white-space changes. What are the actual differences in this file?

@rodw-au
Copy link
Copy Markdown
Contributor Author

rodw-au commented May 3, 2026

Thanks for the feedback.

  1. I will remove the spurious (empty) arc_heading
  2. The nc_folder was created due to issue RIP build now makes a temp file or folder (I think called $) in the working directory where linuxcnc 2.10pre is invoked from the command line #3924 . I reported which was not resolved until after I commenced work on this PR. I will remove it as there is no need to demonstrate these pins with a custom sim. (I just used the normal axis sim for testing)
  3. docs/man/man9/output_buffer.9 I think appeared on acceptance of a PR from Luca so not sure how to remove it. It seems to be created again in the build process and is not part of this feature. Added to docs/man/.gitignore
  4. Sorry if the style has been broken. Changes in interp_convert.cc are two new procedures at the end tag_straight() and tag_arcs() which are each called once from convert_straight() and convert_arc2(). New procedures are from line 7080 and on. Probably easiest to review in my fork; state_tags_arcs branch. https://github.com/rodw-au/linuxcnc/blob/state-tags-arcs/src/emc/rs274ngc/interp_convert.cc#L7080

Ideas to resolve Item 3 would be appreciated.

Just reviewing further, it appears #3924 that the ~/linuxcnc/configs folder contents have been added to the root folder on my PC. I don't think they have been included in this PR. Could someone please check and confirm?

…that was not resolved when work commenced on this PR.

2. Deleted spurious arc_heading file.
3. Moved docs/man/man9/output_buffer.9 to docs/man/.gitignore
@hansu
Copy link
Copy Markdown
Member

hansu commented May 3, 2026

docs/man/man9/output_buffer.9 I think appeared on acceptance of a PR from Luca so not sure how to remove it. It seems to be created again in the build process and is not part of this feature. Added to docs/man/.gitignore

Sorry my fault, I forgot to add the generated man page to gitignore (#3941)

@BsAtHome
Copy link
Copy Markdown
Contributor

BsAtHome commented May 3, 2026

Hm, I'm not sure I've been looking too good at the nc directory... Point 12 makes 300+ files disappear. That seems very, very wrong. (edit should read point 2)

Point 3 should be a separate PR, I guess.

Point 4 needs to be solved by you taking the original file and only change what is necessary.

@BsAtHome
Copy link
Copy Markdown
Contributor

BsAtHome commented May 3, 2026

Just to add a comment on nc files.

It is not that you cannot or should not add (example) files. The point is that there needs to be some kind of docs or procedures that use or reference them. Just having files without people knowing how to find them is kinda pointless.

@BsAtHome
Copy link
Copy Markdown
Contributor

BsAtHome commented May 3, 2026

docs/man/man9/output_buffer.9 I think appeared on acceptance of a PR from Luca so not sure how to remove it. It seems to be created again in the build process and is not part of this feature. Added to docs/man/.gitignore

Sorry my fault, I forgot to add the generated man page to gitignore (#3941)

You are not the first who gets bitten by this. There is a change in the works to let CI fail when this happens.

@rodw-au
Copy link
Copy Markdown
Contributor Author

rodw-au commented May 3, 2026

Just to add a comment on nc files.

It is not that you cannot or should not add (example) files. The point is that there needs to be some kind of docs or procedures that use or reference them. Just having files without people knowing how to find them is kinda pointless.

As mentioned in the original comment, these new pins are documented in the motion man file eg man motion under I_Interpreter Metadata pins._ These should find hteir way to the web site shouldn't they?

Re point 2, you now see the problem with the nc_files folder caused by #3924 . I too only saw it it when the files were removed by git add. It was very wrong as it created the files that should have been under ~/linuxcnc in whatever was the current working directory. So saving the sim to your desktop created this mess!

@BsAtHome
Copy link
Copy Markdown
Contributor

BsAtHome commented May 3, 2026

As mentioned in the original comment, these new pins are documented in the motion man file eg man motion under I_Interpreter Metadata pins._ These should find hteir way to the web site shouldn't they?

The man page is a reference for the component. These man pages usually do not do a lot of examples. For that we have the other documentation in the docs/src directory, where specific scenarios are described and more in depth explanations are (should be) presented. This is usually also the place, while explaining and going into depth, to reference the examples provided.

Re point 2, you now see the problem with the nc_files folder caused by #3924 . I too only saw it it when the files were removed by git add. It was very wrong as it created the files that should have been under ~/linuxcnc in whatever was the current working directory. So saving the sim to your desktop created this mess!

Yes, there was that unfortunate wrong side-effect while fixing the Tcl9 compatibility problem, sorry about that. However, it should be fixed now, isn't it?

@BsAtHome
Copy link
Copy Markdown
Contributor

BsAtHome commented May 3, 2026

These should find hteir way to the web site shouldn't they?

Man pages should find their way, yes. So does the other documentation afaik.
(if auto updating works, don't know about that)

rod added 2 commits May 4, 2026 07:47
in preparation to fix format errors caused by editor auto format options
Aditional commit to follow
then reapplying chnagfes in this PR to it
@rodw-au
Copy link
Copy Markdown
Contributor Author

rodw-au commented May 3, 2026

Formatting issue in interp_convert.cc was caused by an errant autoformatting editor I had no idea about.
I reverted to a copy from master branch than reapplied the changes from this PR so the last commit 5461c2d now shows changes nice and clearly.

I can't see a clean way to correct the .gitignore issue by a PR from the oversight in #3941. Its been corrected in this PR.

Re documentation, when I look at a the docs page on the web site, I can't see any chapter where additional documentation of a few extra pins is appropriate except for the man page. Perhaps these gcode samples could be deployed and mentioned on the man page but where do I put them so they turn up in the users nc_file examples?
nc_files.zip.

Adding example gcode for read only pins just does not seem right to me. The man page should be sufficient (as it is for every other component).

All runtests pass
Runtest: 278 tests run, 278 successful, 0 failed + 0 expected, 4 skipped, 0 shmem errors

I think this is now OK for approval. It is benign as it does not add any additional features, just exposes some useful information. Breakages are unlikely. Any fixes following user feedback can be added.

@rodw-au
Copy link
Copy Markdown
Contributor Author

rodw-au commented May 7, 2026

AFAIK it was reluctance to explicitly write out the serialization stuff for NML but you are right that's a different cake.

Still todo.
Today was a fairly big body of work enabling full supports for all planes.

@rmu75
Copy link
Copy Markdown
Collaborator

rmu75 commented May 7, 2026

Still todo.
Today was a fairly big body of work enabling full supports for all planes.

Don't bother with the NML stuff, that is a different thing, the int and float arrays in the state_tags should remain (for now).

@rodw-au
Copy link
Copy Markdown
Contributor Author

rodw-au commented May 7, 2026

Don't bother with the NML stuff, that is a different thing, the int and float arrays in the state_tags should remain (for now).
nc_files.zip

Thanks.
Then its complete!
I just pushed a correction for the failing RTAI test
Runtest: 278 tests run, 278 successful, 0 failed + 0 expected, 4 skipped, 0 shmem errors

Some simple gcode files in the attached .zip file if anybody wants to test it. Just put all the motion.interp.* pins into halshow and observe the values while running
Files in Zip are:
headings-test1.ngc - arcs on XY plane (G17)
headings-test2.ngc - straight lines on XY plane (G17)
headings-test1-g18.ngc - Arcs on ZX plane (G18)
headings-test1-g19.ngc - Arcs on YZ plane (G19)
helix.ngc (simple helix to show iscircle pin will not go true becasue circular motion not restricted to a single plane)

@BsAtHome
Copy link
Copy Markdown
Contributor

BsAtHome commented May 7, 2026

Would it be possible to add a test under, lets say, tests/motion/heading/ and verify that the code is and remains functional?

@rodw-au
Copy link
Copy Markdown
Contributor Author

rodw-au commented May 7, 2026

I'll have a think about it tomorrow. You could probably test straight headings as they are set at the beginning of a segment but that's not really helpful as you really want to check the arcs which are dynamic. I wonder if you could use hal streamer? A fail would be where the heading does not change during the run. Could that be done?

@rodw-au
Copy link
Copy Markdown
Contributor Author

rodw-au commented May 8, 2026

I'm a bit stuck here. I have the test mostly working, Linuxcnc opens, the gcode file is processed but I am not seeing any of the halsampler output in the results file. If someone could test and update or advise what to do, it would be much appreciated.

Steps Outstanding:

  1. obtain halsampler output
  2. write checkresult script

@rodw-au
Copy link
Copy Markdown
Contributor Author

rodw-au commented May 8, 2026

OK, after a bit more testing with the hal file on a normal axis config, halsampler immediately outputs a buffer full of readings before the GUI opens which means that the buffer is fully depleted before the gcode file is even loaded!
I guess the reason for this is the enable pin is set by default which must be before the digital output I used to enable the sampler is set up.

Unless this can be resolved, its not possible to implement a runtest for the heading. Any ideas please let me know

@BsAtHome
Copy link
Copy Markdown
Contributor

BsAtHome commented May 8, 2026

It may be appropriate to add more pins to the streamer:

  • motion.interp.line-number
  • motion.interp.motion-type
  • motion.interp.feedrate
  • motion.interp.heading

Then instantiate the sampler in HAL (could be a normal HALFILE):

loadrt sampler depth=400 cfg=sfff
net linenum  motion.interp.line-number => sampler.0.pin.0
net mottype  motion.interp.motion-type => sampler.0.pin.1
net feedrate motion.interp.feedrate    => sampler.0.pin.2
net heading  motion.interp.heading     => sampler.0.pin.3

loadusr -w halsampler -n 10000
# Do not enable the sampler or it will start to fill up before we are actually doing something

In the UI script:

...
# Load program
c.program_open("heading-test.ngc")

# Enable sampling
h.set_p("sampler.0.enable", "1");

# Run program
c.auto(linuxcnc.AUTO_RUN, 1)
...

The UI script should wait for the halsampler executable to finish or it must send a SIGTERM to halsample to instruct it to terminate.

@BsAtHome
Copy link
Copy Markdown
Contributor

BsAtHome commented May 8, 2026

Just came to think of it...

I actually fixed the python HAL stream interface. You don't need the halsampler program and could read the data directly in the UI wrapper in its loop.

sampler = h.stream(h, h.sampler_base, "sfff")
...
# sampler.read() returns a tuple with the sample value
def read_samples():
  while sampler.readable:
    print(sampler.read()) # maybe this should do something else...
...

# Don't wait, loop with sampling
# l.wait_for_interp_state(linuxcnc.INTERP_IDLE)
while True:
  s.poll()
  if s.interp_state == linuxcnc.INTERP_IDLE:
    break
  if sampler.readable:
    read_samples()
  time.sleep(0.001)

h.set_p("sampler.0.enable", "0")
time.sleep(0.1)
read_samples() # drain rest
...
# Do other stuff

@rodw-au
Copy link
Copy Markdown
Contributor Author

rodw-au commented May 8, 2026

Thanks, I decided to stick with halsampler for now, then try your new method but I am having a bit of trouble with this

Traceback (most recent call last):
  File "/home/rod/devt/linuxcnc/tests/motion/heading/./test-ui.py", line 47, in <module>
    h.set_p("sampler.0.enable", "1")
    ^^^^^^^
AttributeError: Pin 'set_p' does not exist

And yes, I saw your errant C ; line ending. :)

I did have a lot of other pins which I had tested in a normal sim but was taking baby steps :)

loadrt sampler depth=10000 cfg=ffffffffb

net x-pos-fb joint.0.pos-fb             => sampler.0.pin.0
net y-pos-fb joint.1.pos-fb             => sampler.0.pin.1
net heading motion.interp.heading       => sampler.0.pin.2
net normal motion.interp.normal-heading => sampler.0.pin.3
net radius motion.interp.arc-radius     => sampler.0.pin.4
net center-x motion.interp.arc-center-x => sampler.0.pin.5
net center-y motion.interp.arc-center-y => sampler.0.pin.6
net center-z motion.interp.arc-center-z => sampler.0.pin.7
net iscircle motion.interp.iscircle     => sampler.0.pin.8

The line number is a good idea though.

@BsAtHome
Copy link
Copy Markdown
Contributor

BsAtHome commented May 8, 2026

You import hal, so it should read hal.set_p(...).

Those semicolons... well, habits die hard ;-)

Using halsampler makes it more difficult because it is a program running asynchronous from the UI wrapper and you have to control its termination. Doing the sampler read in the UI wrapper eliminates that async and external program problem go away. But, sure, test it first ;-)

@rodw-au
Copy link
Copy Markdown
Contributor Author

rodw-au commented May 9, 2026

using the old hal way failed as the test-ui never returned so I went to the python method.
I had a number of issues

  1. I could not load the sampler using your syntax and resorted to some AI which suggested I use the numeric instance which worked (maybe)
  2. The sampler is not returning data in Python
  3. Homing was only doing one axis before failing so I changed how we do that

I ran up a normal linuxcnc instance for testing and debugging and the script runs to completion and I could see it homing, loading and running the gcode file. But there is no sampler data arriving adding a few more print statements

I'm really out of my depth here. I've never used the python interface before and barely know the python language. I much prefer ;'s too :)

If somebody could review this after my last commit, it would be great! If we could get data printing, I should be able to devise some tests

@BsAtHome
Copy link
Copy Markdown
Contributor

BsAtHome commented May 9, 2026

I'll have a look.

@BsAtHome
Copy link
Copy Markdown
Contributor

BsAtHome commented May 9, 2026

I have a sampling version of the test. No checks performed on it yet.

The attached zip is the content of the tests/heading directory: heading.zip

You need to git rm the .gitignore and sampler.hal files. There are not necessary.

Problems found:

  • The DISPLAY program needs to be a shell findable path and must be specified with leading ./ (very annoying, I know)
  • You had two HAL files for the sampler setup with different content. Very confusing. Now there is only one (heading.hal) and that one is referenced in the INI-file.
  • The test-ui.py script needed some tweaking to make it work as expected.
  • I added a feed move to the G-code, just to see what happened

The data is currently dumped in the result file. This may not be appropriate and the test-ui.py script needs some updating to manage the data.

Please note that the motion-type field lags one sample when the next command is executed.

@rodw-au
Copy link
Copy Markdown
Contributor Author

rodw-au commented May 9, 2026

Thanks so much for this. Sorry, I thought I had deleted the extra files.
I have been out tonight so will look in the morning.
There could be a lot of data as currently we've just tested a single plane. I need to review the data set but I was thinking I might check say every 200th reading on each line of gcode and compare reported vs calculated data then write out "pass" or "fail" in results. For G18/G19, I probably only need to check the x,y,z centres are correct.

… as we got

inconsistant calculations.

Also have to complete the validation
@rodw-au
Copy link
Copy Markdown
Contributor Author

rodw-au commented May 10, 2026

So had some time today. I have the tests validating now but initially had issues with radius calculations when the machine had imperial units and my gcode was in metric so the tests found an issue before they were released! Updating the ini file in the tests fixed that and all tests passed. I take every 100th entry and perform tests on them as there are 1000s of entries. All lines assessed now pass including g2,g3,g1,g0
I will look into that. Ideally it should respect the gcode measurement system, not device units.

@BsAtHome
Copy link
Copy Markdown
Contributor

So had some time today. I have the tests validating now but initially had issues with radius calculations when the machine had imperial units and my gcode was in metric so the tests found an issue before they were released! Updating the ini file in the tests fixed that and all tests passed.

Yes, the units were off. Also, the feed rate units do probably not match.

I take every 100th entry and perform tests on them as there are 1000s of entries.

That is a problem. This is coming from RT and you do not know how many samples you will get and therefore not reproducible. You probably need to assess the situation whenever the state changes.

@rodw-au
Copy link
Copy Markdown
Contributor Author

rodw-au commented May 10, 2026

result.txt
Now checks every reading

I have the test operating succesfully see reults.txt
I don't understand how to use checkresult. I can't use expected because of the variable timing data linuxcnc returns
I thought I'd be able to open results and search for the "Completed sucessfully" but it is not found
I did see one example that seemed to look for
Runtest: 1 tests run, 0 successful, 1 failed + 0 expected, 0 skipped, 0 shmem errors
but I could not understand to apply it to my situation
I know test.sh will recieve a return code from linuxcnc but I did not see how to apply that
A bit of help to close this out would be great.

Please note that unexpected deviations from the expected path set in the gcode by joint.0.pos-fb joint.0.pos-fb are out of scope of this PR. I noted that around the transition between the G2 and the G1 you added, the radius gradually deviates up to 1.2mm at the end and no G64 or G61 settings would alter this. This was for up to 40 readings at the end of the G2 and 5 readings at the start of the G1

In the end, I surmised that the planner must blend the trajectory regardless of the expected gcode controls so I ignore the deviations in the G2 unless they exceed 40 and skip the first 5 readings in the G1

I think this is a valid approach becasuse the remaining 2700 odd readings gave perfect results and we don't touch anything with the actual motion.

I still need to review gcode vs machine units but wanted to finish this first

After the DISPLAY exits, content is flushed to the resuilt file.
So we overwite this in test.sh based on sucess or fail.
This lets us use an expected file to check pass or fail
@rodw-au
Copy link
Copy Markdown
Contributor Author

rodw-au commented May 11, 2026

Runtests now works.
Comments in the commit

global nsamples, lastsample
sample = sampler.read()
if sample != lastsample: # Don't print duplicates
if (nsamples % SAMPLEN == 0):
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SAMPLEN is one (1) and should always be one. This modulo/if code is therefore noise and should be removed

Comment on lines +125 to +126
if sample[0] not in (4, 5, 6):
# We are only interested in lines 4,5,6. Do not change gcode file
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure you should actually test on line-number. You have the motion-type field (trailing by one sample) that should be enough? If not, why not?

In principle you want to check heading for your motion types and could do so based on that field. It would certainly simplify this code if I'm not mistaking. You can then also change the g-code to suit you without having to hack this code as well.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, we can probably get rid of the line number test and only test G0,1,2,3 lines based on motion type. Lines was my first idea when I had no clue where to start. I've just completed an update to check gcode units vs machine units but have not tested it. I also want to add tests using imperial machine units. (and rebase to master).

One crazy Idea I had was to write back the current heading to a next heading field in the previous segment. This would allow a component to check if a tangential knife needed to be lifted before the next segment and use external offsets before the knife is twisted while still in the material. Would there be a way to do that? (would make it a seperate PR)

@andypugh
Copy link
Copy Markdown
Collaborator

I have the test operating succesfully see reults.txt I don't understand how to use checkresult. I can't use expected because of the variable timing data linuxcnc returns I thought I'd be able to open results and search for the "Completed sucessfully"

I have seen at least one test that pre-processes the "result" file to leave only the pertinent data before comparing.
Here is one example:

# so that a single `expected` file works for both.

@rodw-au
Copy link
Copy Markdown
Contributor Author

rodw-au commented May 11, 2026

I have the test operating succesfully see reults.txt I don't understand how to use checkresult. I can't use expected because of the variable timing data linuxcnc returns I thought I'd be able to open results and search for the "Completed sucessfully"

I have seen at least one test that pre-processes the "result" file to leave only the pertinent data before comparing. Here is one example:

# so that a single `expected` file works for both.

Thanks Andy, I've now done something similar this morning by overwriting the results file with success/failure messages from tests.sh. Thinking about it, I probably could enable verbose messaging so that if the test fails, runtests -n would leave the error messages and test data for review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants