Fix time series preprocess

[CUHKSZzxy]

1. Data url (base64)

In this case, users use a util function from lmdeploy to encode raw data into base64 format and put it into the message.
The input of encode_time_series_base64 can be http url, local file path, file url, or even directly numpy array.

from lmdeploy.vl.time_series_utils import encode_time_series_base64
base64_ts = encode_time_series_base64("0068636_seism.npy")

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "text",
                "text": "Please determine whether an Earthquake event has occurred in the provided time-series data. If so, please specify the starting time point indices of the P-wave and S-wave in the event."
            },
            {
                "type": "time_series_url",
                "time_series_url": {
                    "url": f"data:time_series/npy;base64,{base64_ts}",
                    "sampling_rate": 100
                },
            },
        ],
    }
]

2. HTTP url

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "text",
                "text": "Please determine whether an Earthquake event has occurred in the provided time-series data. If so, please specify the starting time point indices of the P-wave and S-wave in the event."
            },
            {
                "type": "time_series_url",
                "time_series_url": {
                    "url": "https://raw.githubusercontent.com/CUHKSZzxy/Online-Data/main/0068636_seism.npy",
                    "sampling_rate": 100
                },
            },
        ],
    }
]

3. File url

without file:// prefix

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "text",
                "text": "Please determine whether an Earthquake event has occurred in the provided time-series data. If so, please specify the starting time point indices of the P-wave and S-wave in the event."
            },
            {
                "type": "time_series_url",
                "time_series_url": {
                    "url": "/nvme1/zhouxinyu/lmdeploy_dev/0068636_seism.npy",
                    "sampling_rate": 100
                },
            },
        ],
    }
]

with file:// prefix

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "text",
                "text": "Please determine whether an Earthquake event has occurred in the provided time-series data. If so, please specify the starting time point indices of the P-wave and S-wave in the event."
            },
            {
                "type": "time_series_url",
                "time_series_url": {
                    "url": "file:///nvme1/zhouxinyu/lmdeploy_dev/0068636_seism.npy",
                    "sampling_rate": 100
                },
            },
        ],
    }
]

[Copilot]

Pull request overview

Adds and wires up time-series loading/preprocessing support for VL/serving flows (including time_series_url message items), and fixes a small docstring typo.

Changes:

• Add lmdeploy/vl/time_series_utils.py with helpers to load time-series from HTTP/local/data-URL sources and encode/decode base64.
• Update InternS1Pro vision model preprocessing to collect and process time-series inputs directly.
• Extend multimodal server-side message conversion to transform time_series_url items into in-memory time_series arrays, and export load_time_series from lmdeploy.vl.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
lmdeploy/vl/utils.py	Fix typo in encode_image_base64 docstring.
lmdeploy/vl/time_series_utils.py	New utilities for time-series load/encode/decode across multiple sources.
lmdeploy/vl/model/interns1_pro.py	New time-series preprocessing/token-count computation; updated preprocessing path for time series.
lmdeploy/vl/model/base.py	Add collect_time_series helper to extract time-series items from messages.
lmdeploy/vl/init.py	Export load_time_series from lmdeploy.vl.
lmdeploy/serve/processors/multimodal.py	Convert time_series_url inputs into time_series arrays during async multimodal conversion; update multimodal type detection.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

time_series_url.url is loaded server-side via load_time_series(url) and supports file:// and raw local paths. In an API server context this can allow clients to read arbitrary server-local files (and also introduces SSRF risk for http(s) URLs). Consider restricting allowed schemes by default (e.g., only data: and http(s)), and gating local-file access behind an explicit configuration flag.