Skip to content

Update and improve util.py CLI script#14364

Open
talflon wants to merge 14 commits intoCharcoal-SE:masterfrom
talflon:improve-util-script
Open

Update and improve util.py CLI script#14364
talflon wants to merge 14 commits intoCharcoal-SE:masterfrom
talflon:improve-util-script

Conversation

@talflon
Copy link
Copy Markdown

@talflon talflon commented Apr 23, 2025

1. What does your pull request change or introduce to the project?

It updates the util.py CLI so that it:

  • no longer errors on Python 3.11+
  • supports more commands than just bisect
  • in interactive mode, takes the entire line as one argument instead of multiple arguments
  • is runnable in multiple ways via command line options
    • interactive (forceable with -i)
    • via command line arguments, as one line or passing multiple arguments to a single command (-S)
    • via files or pipes (-A, -C), single line or multiline (-0)

To accomplish this, the following changes were made outside of util.py:

  • normalize_number() and deobfuscate_number() in chatcommands: instead of assuming msg is not None, these methods now treat msg=None similar to how bisect() and bisect_number() already do, by falling back to the passed-in pattern argument
  • make Post initializable with ms_api_response, adding corresponding method _parse_ms_api_post()
  • add Metasmoke.get_post_from_ms() which loads a Metasmoke post given its id or URL, and turns it into a Post

The following new commands call into the same internal code as their namesake chat commands:

  • bisect-number
  • deobfuscate-number
  • normalize-number
  • test-question / test-q
  • test-answer / test-a
  • test-user / test-u
  • test-title / test-t
  • test
  • test-json / test-j

The following new commands load posts and test them:

  • scan: takes an SE URL
  • test-ms: takes an MS URL

The following new commands create temporary copies of the relevant blacklists, then manually add or remove items from these temporary copies, so that subsequent calls to bisect-* or test-* commands will test these modifications. They don't run the full logic of their namesake chat commands:

  • blacklist-keyword
  • blacklist-website
  • blacklist-username
  • blacklist-number
  • unblacklist

2. What is the justification for the inclusion of your Pull Request (or, what problem does it solve?)

It lets someone test the blacklists and most of the other spam rules outside of public chat, by cloning the SmokeDetector repository and installing the dependencies. If they want to test a lot of different things, it avoids spamming a chatroom. Furthermore, they could use this to test new regexes using the actual SmokeDetector code, before creating a commit on the public repository, or use this to test new spam detection code without starting up a full SmokeDetector instance.

util.py used to be able to do this for bisect, but that was broken due to the removal of inspect.getargspec() in Python 3.11. I fixed that, and extended the functionality of this script.

3. Write meaningful commit messages ...

I was pretty brief in my commit messages. In the later ones I tried to explain a little more.

4. Include comments in your code ...

I did some of that, where I didn't think it was self-explanatory.

5. Use meaningful variable names ...

I've tried to do that.

6. What testing have you done?

I have only tested it by running util.py myself and playing around with it. util.py wasn't working for me before I started, so I'm hoping I can't have broken it too badly.

I believe the only way it could introduce errors in other parts of SmokeDetector is if there was code relying on deobfuscate_number() or normalize_number() raising an error on msg=None. I don't see any error handling code related to that, so I'm hoping there's no requirement for that to raise.

Daniel Getz added 10 commits June 9, 2025 17:16
- if it can be single line, run it as single line
- detect multi-line input with starting quotes, and handle similar to before
- no longer exit on a blank interactive input line
- still allows command line arguments, but now allows them to be treated individually
  or as a single line with -S
- force interactive mode with -i after running other commands
- specfify command more explicitly with -c
- load commands from a file/pipe with -C
- load arguments to the specified command from a file/pipe with -A
- allows null-terminating input files
  for files without any null bytes, that means treating the file as a single argument
  or command
@talflon talflon force-pushed the improve-util-script branch from 2f2efa5 to 5d52b94 Compare June 9, 2025 21:16
Daniel Getz added 4 commits June 9, 2025 17:38
- loads a post from SE API and then tests it
- loads a post from a Metasmoke URL and re-runs current detection on it
- add Post._parse_ms_api_post, Post(ms_api_response=)
- add Metasmoke.get_post_from_ms()
@talflon talflon added the type: enhancement Improvements which don't reach the level of being new features. label Jun 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type: enhancement Improvements which don't reach the level of being new features.

Development

Successfully merging this pull request may close these issues.

1 participant