Task Evaluation Criteria
Configure custom success and failure standards for task tracking
Define what "success" and "failure" mean for your project's tasks. By default, Acontext uses generic criteria — configure custom ones to match your agent's domain (e.g. customer support, coding, research).
Configure via SDK
import os
from acontext import AcontextClient
client = AcontextClient(api_key=os.getenv("ACONTEXT_API_KEY"))
# Set custom task evaluation criteria
client.project.update_configs({
"task_success_criteria": "The customer's issue is fully resolved and they confirm satisfaction, or the agent provides a verified solution with supporting documentation.",
"task_failure_criteria": "The agent cannot resolve the issue after exhausting available options, the customer explicitly reports dissatisfaction, or the conversation is abandoned without resolution.",
})
# View current config
configs = client.project.get_configs()
print(configs)import { AcontextClient } from '@acontext/acontext';
const client = new AcontextClient({
apiKey: process.env.ACONTEXT_API_KEY,
});
// Set custom task evaluation criteria
await client.project.updateConfigs({
task_success_criteria: "The customer's issue is fully resolved and they confirm satisfaction, or the agent provides a verified solution with supporting documentation.",
task_failure_criteria: "The agent cannot resolve the issue after exhausting available options, the customer explicitly reports dissatisfaction, or the conversation is abandoned without resolution.",
});
// View current config
const configs = await client.project.getConfigs();
console.log(configs);Reset to Defaults
Pass null to remove a custom criteria and revert to the built-in default.
client.project.update_configs({
"task_success_criteria": None,
"task_failure_criteria": None,
})await client.project.updateConfigs({
task_success_criteria: null,
task_failure_criteria: null,
});Configure via Dashboard
You can also set task evaluation criteria from the Dashboard UI.
Go to Project Settings → Task Tracking tab. Edit the Task Success Criteria and Task Failure Criteria text fields, then click Save.
Go to Settings in the sidebar. Edit the Task Success Criteria and Task Failure Criteria text fields, then click Save.
Custom criteria are injected into the task agent's input prompt. The system prompt stays identical across all projects, so LLM prompt caching (Anthropic and OpenAI) is fully preserved regardless of your custom criteria.
Next Steps
Last updated on