Guide: Stream 1.7M AgentTrove Interaction Traces and Build a ShareGPT-Format SFT Dataset in Python

A practical Python tutorial demonstrates streaming access to open-thoughts/AgentTrove, an open-source collection of about 1.7 million agentic interaction rows in a ShareGPT — style layout, and shows how to turn sampled traces into a clean ShareGPT — format JSONL suitable for supervised fine-tuning (SFT). for researchers and practitioners working with large interaction logs.

The post lists concrete setup and tooling: pip-install instructions for datasets>=2.19, pandas, matplotlib, pyarrow and huggingface_hub; it sets REPO = 'open-thoughts/AgentTrove' and uses datasets.load_dataset(..., streaming=True) to access data without downloading the entire collection. These steps enable immediate, memory — efficient sampling and inspection of traces directly from the Hugging Face datasets API.

To parse and standardize interactions the tutorial introduces utility functions such as find_trace_key() to locate the conversation column and normalize_turns() to standardize role and content fields. It also includes parsers for command — style assistant outputs and routines to render and sample trajectories, letting users extract coherent turn sequences from messy assistant traces.

The workflow then converts sampled traces into pandas DataFrames, produces turn-level summaries and visualizations with matplotlib, and exports selected successful trajectories into a clean ShareGPT — style JSONL for SFT. Those export routines are designed to support reproducible dataset curation and faster experimentation by producing consistent, training — ready records from raw agent traces.