Skill Activity

Skills are activated/triggered by the agent based on the contents of the skill's description. Under- or over-specified descriptions will not get triggered — you need to get your description just right.

Agents use progressive disclosure to manage context: at startup they load only the name and description of each available skill. When a user's task matches a description, the agent reads the full SKILL.md. Agents only consult skills for tasks beyond their knowledge or capabilities — even if a task matches perfectly, it will not trigger if the agent can do it without the skill.

Optimizing a description for activation:

1. Weak: "This skill handles data pipelines. It knows about ETL processes, transformation jobs, and orchestration tools like Airflow and dbt."
2. Frame the description as an instruction to the agent: "Use this skill when the user wants to build, debug, or optimize data pipelines..."
3. Describe what the user is trying to achieve, not the skill's internal mechanics: "Use this skill when the user wants to move data between systems, schedule batch jobs, or make raw data usable for downstream consumers..."
4. Explicitly list contexts where the skill applies: "...even if they don't explicitly mention 'ETL' or 'pipeline.' Trigger on mentions of Airflow, dbt, Prefect, Spark, ingestion jobs, or data freshness problems. Also trigger when the user describes symptoms like 'my data is stale'..."
5. Keep it concise — the specification enforces a hard limit of 1024 characters.

Evaluation: to test your skill you can create a JSON file with example queries, iterate over them calling the API and recording the results.

• Recommend to create 20 queries: 10 that should trigger, 10 that shouldn't trigger.
• Recommend to run all queries at least 3x.
• Remember the 60/40 rule for Train vs Validation test sets.
• Try not to include all possible queries in your eval_queries.json.
• A score of over 0.5 trigger rate should be considered a successful skill.
• There's no prescribed implementation — you'll need to code it yourself for your API.

Example eval_queries.json:

[
  {
    "query": "I've got a spreadsheet in ~/data/q4_results.xlsx with revenue in col C and expenses in col D — can you add a profit margin column and highlight anything under 10%?",
    "should_trigger": true
  },
  {
    "query": "whats the quickest way to convert this json file to yaml",
    "should_trigger": false
  }
]

Related: Agent Skills, Skill Anatomy