Please replace the use of unmaintained module common_io

[yurivict]

It's PYPI page https://pypi.org/project/common-io-python/ is questionable:

• The package description says "h4ck3d". Is it really hacked?
• Homepage link points to the image of cheese.
• No description, no documentation
• N source code
• last updated 5+ years ago

I can't create a FreeBSD port for it for faiss tests to run.

Please consider replacing this questionable module.

[mnorris11]

Hey @kyamagu do you know why this module is included? We will work on ramping up on this. @alibekfc is taking a look next week.

[kyamagu]

The common_io module seems to be imported only within the tests, and the index_io_backward_compatibility test package was recently introduced. I am not aware of the context of this module. The earliest relevant PR seems #4706

% grep -R common_io
./tests/index_io_backward_compatibility/conda_writer.py:from common_io import write_test_all_files
./tests/index_io_backward_compatibility/conda_reader.py:from common_io import read_test_all_files
./tests/index_io_backward_compatibility/cmake_writer.py:from common_io import write_test_all_files
./tests/index_io_backward_compatibility/cmake_reader.py:from common_io import read_test_all_files

[mnorris11]

hmm, I created this common_io in our code, but I don't know why PyPI brings in some unrelated module. @yurivict is this what you see, like installing Faiss with pypi that it brings in some random common_io module rather than the common_io in our code? Do you have a quick repro? We are still working on official PyPI support (the official way right now is through the Pytorch conda channel) but we should start handling these types of issues to ramp up.

[kyamagu]

It could be that the import statement misses the relative import, or that tests/__init__.py is missing?

Relative

from .common_io import read_test_all_files

Absolute

from common_io import read_test_all_files

[yurivict]

Tests fail to find common_io:

ImportError while importing test module '/usr/ports/math/py-faiss/work-py311/faiss-1.14.0/tests/index_io_backward_compatibility/cmake_writer.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/local/lib/python3.11/site-packages/_pytest/python.py:498: in importtestmodule
    mod = import_path(
/usr/local/lib/python3.11/site-packages/_pytest/pathlib.py:587: in import_path
    importlib.import_module(module_name)
/usr/local/lib/python3.11/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<frozen importlib._bootstrap>:1204: in _gcd_import
    ???
<frozen importlib._bootstrap>:1176: in _find_and_load
    ???
<frozen importlib._bootstrap>:1147: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:690: in _load_unlocked
    ???
<frozen importlib._bootstrap_external>:940: in exec_module
    ???
<frozen importlib._bootstrap>:241: in _call_with_frames_removed
    ???
index_io_backward_compatibility/cmake_writer.py:15: in <module>
    from common_io import write_test_all_files
E   ModuleNotFoundError: No module named 'common_io'
_____________________________________________________ ERROR collecting index_io_backward_compatibility/conda_reader.py ______________________________________________________
ImportError while importing test module '/usr/ports/math/py-faiss/work-py311/faiss-1.14.0/tests/index_io_backward_compatibility/conda_reader.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/local/lib/python3.11/site-packages/_pytest/python.py:498: in importtestmodule
    mod = import_path(
/usr/local/lib/python3.11/site-packages/_pytest/pathlib.py:587: in import_path
    importlib.import_module(module_name)
/usr/local/lib/python3.11/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<frozen importlib._bootstrap>:1204: in _gcd_import
    ???
<frozen importlib._bootstrap>:1176: in _find_and_load
    ???
<frozen importlib._bootstrap>:1147: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:690: in _load_unlocked
    ???
<frozen importlib._bootstrap_external>:940: in exec_module
    ???
<frozen importlib._bootstrap>:241: in _call_with_frames_removed
    ???
index_io_backward_compatibility/conda_reader.py:15: in <module>
    from common_io import read_test_all_files
E   ModuleNotFoundError: No module named 'common_io'

tests/__init__.py isn't present in the repository.

I initially didn't notice that common_io.py is present in the code. It isn't found, this is the problem.

[mnorris11]

I just released 1.14.1 on conda with this file name changed. Does this issue still occur @yurivict ?