Breaking Change on Copy Command

[tietor]

Hello

I upgraded shx from version 0.3.4 to 0.4.0. I executed my shx command after the upgrade and an error occured with the following command:

shx cp C:\myDirectory\*.scss C:\otherDirectory

This is the exception:
[shx cp "C:\myDirectory\*.scss" "C:\otherDirectory"](cp: no such file or directory: C:/myDirectory/*.scss)

I think the problem in the command is *.scss. I only want to copy the scss Files in the myDirectory folder.

How can I do that in the current verion? This command worked in the 0.3.4 version.

Regards
Fabio

[nfischer]

Can you check the changelog? https://github.com/shelljs/shelljs/releases/tag/v0.9.0. Let me know if you find the change which you think broke your use case.

[tietor]

I couln't find a change which relates to the copy command. May you provide me an example how I can copy scss Files from one directory to my other? It is important, that I can filter by extensions. This is the main problem in the version 0.4.0.

[nfischer]

Can you give a repro case on Linux?

[tietor]

I analyzed it again and I think this problem only occurs on windows with the version 0.4.0. I don't have this problem with the version 0.3.4.

This is my test setup:

1. I created in C drive a folder named copyFiles
2. In this folder I created 3 Files named firstTest.scss, secondTest.scss and myCss.css:

3. create in C drive an empty folder named pasteLocation:

This is how I reproduced the issue:

1. install shx version 0.3.4 globally with this command: npm i shx@0.3.4 -g
2. execute this command to copy the scss files from copyFiles to pasteLocation: shx cp C:\copyFiles\*.scss C:\pasteLocation. This command was executed successfully without an error. The scss files are copied to C:\pasteLocation:

3. install shx version 0.4.0 globally with this command: npm i shx@0.4.0 -g
4. execute again the exact same command from step 2 to copy the scss files files from copyFiles to pasteLocation: shx cp C:\copyFiles\*.scss C:\pasteLocation. This command throws the following exception: cp: no such file or directory: C:/copyFiles/*.scss

Here is a screenshot of my powershell window where you can see the execution of all 4 steps:

[nfischer]

Can you give a repro case on Linux or is this Windows-specific? I would need to know the answer to this in order to advise on what is going on.

[nfischer]

It sounds like this is probably an issue with the new globbing library. Can you try installing the fast-glob package and test out if this has the ability to expand either of the following?

// In a new nodejs repl or in a ".js" file executed with nodejs
const globOpts = {
    onlyFiles: false,
    followSymbolicLinks: false,
};
const ret = fastGlob.sync("C:/copyFiles/*.scss", globOpts);
console.log(ret);
const ret2 = fastGlob.sync("C:\\copyFiles\\*.scss", globOpts);
console.log(ret2);

This is based on how we do globbing in shelljs code:

https://github.com/shelljs/shelljs/blob/013fc0041b15ecc28ef2a3aa19d891a7471ecd87/src/common.js#L310C10-L310C43

[tietor]

Tested with fast-glob version 3.3.3. This is my code:

Here you can find the output from this script:

ret2 returns an empty array.

Do you still need an example in linux?

[nfischer]

If you can repro on Linux, then I can fix the bug. I can't personally support Windows-specific bugs unless there's a clear and obvious change to make, or a unit test that shows the problem.

[tietor]

I found out that my example works when I use frontslashes C:/copyFiles/*.scss instead of 2 backslashes C:\\copyFiles\\*.scss

Can you document this as a windows specific breaking change? I don't think that you need to fix this.

[nfischer]

Thanks for confirming. I think we're supposed to be able to handle backslashes properly, but I'm glad you can use front-slashes as a workaround. I'll keep this bug open for now. I'm going to look into whether I can write a test case in the shelljs project which covers this and see if I can reproduce the bug that way.

[BrendanC23]

This seems to be a limitation of the fast-glob library. From the README:

Always use forward-slashes in glob expressions (patterns and ignore option). Use backslashes for escaping characters. With the cwd option use a convenient format.
[...]
📖 Use the .convertPathToPattern package to convert Windows-style path to a Unix-style path.

Here's a shxjs test that fails on Windows:

test('cp with Windows-style backslash and glob pattern', t => {
  const result = shell.cp('test\\resources\\file*.txt', t.context.tmp);
  t.falsy(shell.error());
  t.falsy(result.stderr);
  t.is(result.code, 0);
  t.truthy(fs.existsSync(`${t.context.tmp}/file1.txt`));
  t.truthy(fs.existsSync(`${t.context.tmp}/file2.txt`));
});

This test passes if this line is changed to the following:

ret = glob.sync(glob.convertPathToPattern(convertQuestionMarkForGlob(listEl)), globOpts);

I'm not sure if this will cause problems in other cases. I tried running all of the existing tests locally but ran into issues with different numbers of tests running each time. There were also many failing tests even without the change.