Skip to content
Open-Source Registry

cargo / turbospark-tokenizer

turbospark-tokenizer 0.3.0

bitbison: clean, with notes, behavior clean, dependencies notes, posture not checked, pipeline clean, 1 of 1 jobs captured clean
processes
176
operations
26K
packages
81
compiler runs
74
connections
1
job time
1m 02s
capture
245 KB
records
42K
bytes / record
5.9

Tokenizer wrapper, chat-template rendering, streaming detokenizer, and tool-call parsing for the supported model dialects.

Build behaviourcompiled C/C++

37 downloads · MIT · source · cargo

Artifact

file · retained · product · graph v2
1
ChecksDetections
Build behavior
✓ Network activity—
✓ Sensitive data access and transfer—
✓ Cross-process access—
✓ Build file changes—
✓ Build script activity—
✓ Data flow analysis—
✓ Filesystem writes—
✓ Context collection—
Supply chain
✓ Known vulnerabilities—
▲ Build-time dependency advisories—
✓ Dependency advisories—
✓ Declared and observed dependencies—
✓ Dependency and toolchain versions—
Pipeline
✓ Pipeline coverage—

Detections

0

No detections

No abnormal activity detected across 26K flows.

Network

0 peers
    cargo fetched 3,910 files from static.crates.io

    Raw data

    capture
    245 KB
    records
    41,809
    bytes per record
    5.9

    Loading a sample of the capture

    Want to see more? E-mail us at hello@bitbison.io about your use case.

    Dependencies

    83 packages, 16 with a build script that ran, 0 doing more than compiling
    PackageVersionUsageBuild behaviour
    crossbeam-deque0.8.7observedbuild script
    crossbeam-epoch0.9.20observedbuild script
    crossbeam-utils0.8.22observedbuild script
    crunchy0.2.4observedbuild script, not run
    darling_macro0.20.11observedproc macro
    derive_builder_macro0.20.2observedproc macro
    esaxx-rs0.1.10observedbuild script
    ld-linux-x86-64.so.2linkedsystem library
    libc0.2.189observedbuild script
    libc.so.6linkedsystem library
    libgcc_s.so.1linkedsystem library
    libm.so.6linkedsystem library
    libmvec.so.1linkedsystem library
    macro_rules_attribute-proc_macro0.2.3observedproc macro
    monostate-impl0.1.18observedproc macro
    onig_sys69.9.3observedbuild script
    paste1.0.15observedbuild scriptproc macro
    pastey0.2.3observedproc macro
    proc-macro21.0.107observedbuild script
    quote1.0.47observedbuild script
    rayon-core1.13.0observedbuild script
    serde direct1.0.229observedbuild script
    serde_core1.0.229observedbuild script
    serde_derive1.0.229observedproc macro
    serde_json direct1.0.151observedbuild script
    thiserror1.0.69observedbuild script
    thiserror-impl1.0.69observedproc macro
    zerocopy0.8.55observedbuild script
    zerocopy-derive0.8.55observedproc macro
    zmij1.0.23observedbuild script
    aho-corasick1.1.5observed
    base640.13.1observed
    bitflags2.13.1observed
    cc1.4.0observed
    cfg-if1.0.4observed
    darling0.20.11observed
    darling_core0.20.11observed
    derive_builder0.20.2observed
    derive_builder_core0.20.2observed
    either1.17.0observed
    find-msvc-tools0.1.9observed
    fnv1.0.7observed
    getrandom0.2.17observed
    half2.7.1observed
    ident_case1.0.1observed
    itertools0.11.0observed
    itertools0.12.1observed
    itoa1.0.18observed
    lazy_static1.5.0observed
    log0.4.33observed
    macro_rules_attribute0.2.3observed
    memchr2.8.3observed
    memo-map0.3.3observed
    minijinja direct2.22.0observed
    minijinja-contrib direct2.22.0observed
    minimal-lexical0.2.1observed
    monostate0.1.18observed
    nom7.1.3observed
    once_cell1.21.4observed
    onig6.5.3observed
    pkg-config0.3.33observed
    ppv-lite860.2.21observed
    rand0.8.7observed
    rand_chacha0.3.1observed
    rand_core0.6.4observed
    rayon1.12.0observed
    rayon-cond0.3.0observed
    regex1.13.1observed
    regex-automata0.4.18observed
    regex-syntax0.8.11observed
    shlex2.0.1observed
    smallvec1.15.2observed
    spm_precompiled0.1.4observed
    strsim0.11.1observed
    syn2.0.119observed
    syn3.0.3observed
    tokenizers direct0.20.4observed
    turbospark-core direct0.3.0observed
    unicode-ident1.0.24observed
    unicode-normalization-alignments0.1.12observed
    unicode-segmentation1.12.0observed
    unicode_categories0.1.1observed
    wasi0.11.1+wasi-snapshot-preview1observed

    1 of 1 jobs captured: build-test success · Generated 2026-10-12 · run ee47fdb7-ce7b-423e-a5e5-e9feebdb44bd

    analysis 0.1.0 · 2026-10-11 · d6246d3 · graph v2