{
  "model_details": {
    "provider": "openai",
    "model_requested": "gpt-5.4-mini",
    "model_for_requests": "gpt-5.4-mini",
    "api_base_url": "https://api.openai.com/v1",
    "chat_completions_endpoint": "https://api.openai.com/v1/chat/completions"
  },
  "run_config": {
    "input": [
      "data/input/like_interrater.csv"
    ],
    "labels": null,
    "task_name": "*like* discourse/pragm",
    "task_description": "Classification of instance of *like* into four discourse-pragmatic categories: 1. Quotative marker, 2. Discourse marker, 3. Clause-medial pragmatic marker, 4. Clause-final pragmatic marker",
    "tags": "*like*;discourse;pragmatics;English;v3",
    "model": "gpt-5.4-mini",
    "temperature": null,
    "top_p": null,
    "top_k": null,
    "service_tier": "flex",
    "verbosity": null,
    "reasoning_effort": null,
    "thinking_level": null,
    "effort": null,
    "strict_control_acceptance": true,
    "provider": "openai",
    "system_prompt": "You are a meticulous linguistic classifier. Classify each instance of _\\*like\\*_ into \\*\\*one\\*\\* of the four discourse-pragmatic categories:\n\n1. Quotative marker \n2. Discourse marker \n3. Clause-medial pragmatic marker \n4. Clause-final pragmatic marker\n \n\nFollow the order of classification precisely as specified: \n\n**Step 1 - Test Omissibility**\nDetermine whether like is omissible without affecting the propositional meaning or grammatical well-formedness of the clause.\n\n* If not omissible, first check for quotative marker use.\n\n**Quotative marker** \nClassify as Quotative marker if like occurs in a quotative frame such as:\n\n* be like, it’s like, verbs of saying or thinking + like (_say, go, think_)\n* like before reported material\n \nand introduces:\n* reported speech, or\n* thoughts, feelings, attitudes, or inner monologue.\n\nExamples: \\* She was like do not tempt me! \\* \\*And I’m like: „I cannot tell them what I am thinking.“ \\* \\* And everyone was like damn. \\* If not, continue. \n\n**Step 2:  Check non-quotative** _**it’s like**\n\n_If _like_ occurs in the construction _it’s like_, is not followed by quoted material, and the whole construction can be omitted without affecting grammaticality or propositional meaning, classify it as a Discourse marker.This construction may function as:\n\n* discourse link (elaboration, clarification, exemplification)\n* topic-orientation marker\n* disfluency marker\n \nExample: \\*He‘s just fired me. It’s like I don’t understand why. \\* \\*It’s like erm uh, I wanted to see you. \\* If _like_ is not part of quotative construction, or discourse marker _it’s like_, and it cannot be omitted without affecting grammaticality and propositional content of its host unit, the use is propositional (verb, noun, preposition, adverb) and should be classified as \"unclassified\". \n\n**Step 3 — If** _**like**_ **is omissible, classify it primarily by position, and use the functional descriptions and examples only to confirm that the positional classification is appropriate.** \n\n**Discourse marker**\nIf _like_ occurs clause-initially or outside clause structure, classify as Discourse marker.Function: organises discourse.Subfunctions:\n\n* discourse link (elaboration, clarification, exemplification)\n* topic orientation (topic shift, digression, topic return)\n* disfluency marker (false start, repair, repetition, pause filler)\n\nExamples: \\*It was really late and like we just left. \\*; \\*I don’t know what happened erm like uh I don’t remember. \\* If not, continue. \n\n**Clause-medial pragmatic marker**\n\nIf _like_ occurs clause-medially, inside the clause before a word or phrase, classifiy as Clause-medial pragmatic marker.Function: scopes forward over following material.Subfunctions:\n\n* marker of non-equivalence between what is said and what was meant:\n* requiring loosening (figurative/non-literal interpretation)\n* requiring enrichment (vague or underspecified expression)\n* focus marker\n* highlights salient information, including intensification or exemplification\n \n\nExamples: \\*That’s like really weird.\\* \\*They only have like bread and onions. \\*\\* I like almost died. \\* \\*It was this like big round thing. \\* If not, continue. \n\n**Clause-final pragmatic marker**\n\nIf _like_ occurs at the end of a clause or utterance, classify as Clause-final pragamtic marker.Function: scopes backward over preceding material.Subfunctions:\n\n* focus marker (clearing up misunderstandings, emphasis)\n* marker of non-equivalence\n* invariant tag\n* checking understanding\n* requesting confirmation\n \n\nExample: \\*He’s a bastard, like. \\* \\* I simply cleaned it with vinegar, like. \\* \\*But I think he already paid, like? \\*",
    "system_prompt_b64": null,
    "few_shot_examples": 0,
    "prompt_layout": "standard",
    "cache_pad_target_tokens": 0,
    "prompt_cache_key": null,
    "gemini_cached_content": null,
    "requesty_auto_cache": null,
    "vertex_auto_adc_login": null,
    "vertex_access_token_refresh_seconds": null,
    "create_gemini_cache": false,
    "gemini_cache_ttl": 3600,
    "gemini_cache_ttl_autoupdate": true,
    "keep_gemini_cache": false,
    "enable_cot": true,
    "no_explanation": false,
    "logprobs": false,
    "calibration": true,
    "confusion_heatmap": true,
    "api_key_var": "OPENAI_API_KEY",
    "api_base_var": "OPENAI_BASE_URL",
    "max_retries": 3,
    "retry_delay": 5.0,
    "request_interval_ms": 0,
    "request_timeout_seconds": 30.0,
    "threads": 5,
    "prompt_log_detail": "full",
    "flush_rows": 100,
    "flush_seconds": 2.0,
    "validator_cmd": null,
    "validator_args": "",
    "validator_timeout": 5.0,
    "validator_prompt_max_candidates": 50,
    "validator_prompt_max_chars": 8000,
    "validator_exhausted_policy": "accept_blank_confidence",
    "validator_debug": false,
    "log_level": "INFO"
  },
  "source_input_csv": "data/input/like_interrater.csv",
  "source_output_csv": "data/output/like_interrater__openai__gpt54mini__2026-03-20-18-10.csv",
  "source_labels_csv": "",
  "cache_padding": {
    "enabled": false,
    "target_shared_prefix_tokens": 0,
    "calibration_shared_prefix_tokens": null,
    "target_prompt_tokens": 0,
    "calibration_prompt_tokens": null,
    "calibration_example_id": null,
    "applied_padding_tokens_estimate": 0,
    "examples_with_padding_applied": 0
  },
  "request_control_summary": {
    "configured": {},
    "attempts_total": 118,
    "attempts_with_control_telemetry": 118,
    "per_control": {
      "reasoning_effort": {
        "configured_value": null,
        "requested_attempts": 0,
        "sent_attempts": 0,
        "accepted_attempts": 0,
        "rejected_attempts": 0,
        "missing_from_final_request_attempts": 0,
        "acceptance_rate": null,
        "rejected_reasons": {},
        "rejected_example_ids": []
      },
      "thinking_level": {
        "configured_value": null,
        "requested_attempts": 0,
        "sent_attempts": 0,
        "accepted_attempts": 0,
        "rejected_attempts": 0,
        "missing_from_final_request_attempts": 0,
        "acceptance_rate": null,
        "rejected_reasons": {},
        "rejected_example_ids": []
      },
      "effort": {
        "configured_value": null,
        "requested_attempts": 0,
        "sent_attempts": 0,
        "accepted_attempts": 0,
        "rejected_attempts": 0,
        "missing_from_final_request_attempts": 0,
        "acceptance_rate": null,
        "rejected_reasons": {},
        "rejected_example_ids": []
      },
      "verbosity": {
        "configured_value": null,
        "requested_attempts": 0,
        "sent_attempts": 0,
        "accepted_attempts": 0,
        "rejected_attempts": 0,
        "missing_from_final_request_attempts": 0,
        "acceptance_rate": null,
        "rejected_reasons": {},
        "rejected_example_ids": []
      },
      "prompt_cache_key": {
        "configured_value": null,
        "requested_attempts": 0,
        "sent_attempts": 0,
        "accepted_attempts": 0,
        "rejected_attempts": 0,
        "missing_from_final_request_attempts": 0,
        "acceptance_rate": null,
        "rejected_reasons": {},
        "rejected_example_ids": []
      },
      "gemini_cached_content": {
        "configured_value": null,
        "requested_attempts": 0,
        "sent_attempts": 0,
        "accepted_attempts": 0,
        "rejected_attempts": 0,
        "missing_from_final_request_attempts": 0,
        "acceptance_rate": null,
        "rejected_reasons": {},
        "rejected_example_ids": []
      },
      "requesty_auto_cache": {
        "configured_value": null,
        "requested_attempts": 0,
        "sent_attempts": 0,
        "accepted_attempts": 0,
        "rejected_attempts": 0,
        "missing_from_final_request_attempts": 0,
        "acceptance_rate": null,
        "rejected_reasons": {},
        "rejected_example_ids": []
      }
    }
  },
  "usage_metadata_summary": {
    "attempts_total": 118,
    "attempts_with_usage_metadata": 118,
    "attempts_with_cached_token_signals": 118,
    "cached_tokens_total_estimate": 0,
    "cache_read_tokens_total": 0,
    "cache_write_tokens_total": 0,
    "cache_token_fields_totals": {
      "usage.prompt_tokens_details.cached_tokens": 0
    },
    "attempts_with_gemini_cached_content_token_signals": 0,
    "gemini_cached_content_token_count_total": 0,
    "gemini_cached_content_token_fields_totals": {}
  },
  "token_usage_totals": {
    "attempts_total": 118,
    "attempts_with_token_usage": 118,
    "attempts_with_output_tokens": 118,
    "attempts_with_cached_input_tokens": 0,
    "attempts_with_thinking_tokens": 0,
    "input_tokens_total": 164791,
    "cached_input_tokens_total": 0,
    "non_cached_input_tokens_total": 164791,
    "output_tokens_total": 13028,
    "thinking_tokens_total": 0,
    "output_tokens_definition": "total_tokens - prompt_tokens (or completion_tokens + thinking_tokens fallback)"
  },
  "truth_label_count": 115,
  "prediction_count": 115,
  "evaluated_example_count": 115,
  "calibration_metrics": {
    "available": true,
    "sample_count": 115,
    "bin_count": 10,
    "ece": 0.33904347826086983,
    "mce": 0.77,
    "brier_score": 0.3413939130434783
  },
  "first_prompt_timestamp": "2026-03-20T17:11:01.726403Z",
  "last_prompt_timestamp": "2026-03-20T17:11:32.800639Z",
  "overall_time_seconds": 31.074236,
  "overall_time_human": "31s",
  "accuracy": 0.6086956521739131,
  "macro_precision": 0.3938311688311688,
  "macro_recall": 0.33125,
  "macro_f1": 0.3320611577964519,
  "per_label": {
    "Clause-final pragmatic marker": {
      "precision": 1.0,
      "recall": 0.44,
      "f1": 0.6111111111111112,
      "support": 25
    },
    "Clause-initial pragmatic marker": {
      "precision": 0.0,
      "recall": 0.0,
      "f1": 0.0,
      "support": 0
    },
    "Clause-internal pragmatic marker": {
      "precision": 0.0,
      "recall": 0.0,
      "f1": 0.0,
      "support": 0
    },
    "Clause-medial pragmatic marker": {
      "precision": 0.43636363636363634,
      "recall": 0.96,
      "f1": 0.6,
      "support": 25
    },
    "Discourse marker": {
      "precision": 0.7142857142857143,
      "recall": 0.5,
      "f1": 0.588235294117647,
      "support": 40
    },
    "Quotative marker": {
      "precision": 1.0,
      "recall": 0.75,
      "f1": 0.8571428571428571,
      "support": 20
    },
    "quotative marker": {
      "precision": 0.0,
      "recall": 0.0,
      "f1": 0.0,
      "support": 0
    },
    "unclassified": {
      "precision": 0.0,
      "recall": 0.0,
      "f1": 0.0,
      "support": 5
    }
  },
  "labels": [
    "Clause-final pragmatic marker",
    "Clause-initial pragmatic marker",
    "Clause-internal pragmatic marker",
    "Clause-medial pragmatic marker",
    "Discourse marker",
    "Quotative marker",
    "quotative marker",
    "unclassified"
  ],
  "label_count": 8,
  "total_examples": 115,
  "confusion_matrix": {
    "Clause-final pragmatic marker": {
      "Clause-final pragmatic marker": 11,
      "Clause-initial pragmatic marker": 0,
      "Clause-internal pragmatic marker": 0,
      "Clause-medial pragmatic marker": 6,
      "Discourse marker": 6,
      "Quotative marker": 0,
      "quotative marker": 0,
      "unclassified": 2
    },
    "Clause-initial pragmatic marker": {
      "Clause-final pragmatic marker": 0,
      "Clause-initial pragmatic marker": 0,
      "Clause-internal pragmatic marker": 0,
      "Clause-medial pragmatic marker": 0,
      "Discourse marker": 0,
      "Quotative marker": 0,
      "quotative marker": 0,
      "unclassified": 0
    },
    "Clause-internal pragmatic marker": {
      "Clause-final pragmatic marker": 0,
      "Clause-initial pragmatic marker": 0,
      "Clause-internal pragmatic marker": 0,
      "Clause-medial pragmatic marker": 0,
      "Discourse marker": 0,
      "Quotative marker": 0,
      "quotative marker": 0,
      "unclassified": 0
    },
    "Clause-medial pragmatic marker": {
      "Clause-final pragmatic marker": 0,
      "Clause-initial pragmatic marker": 0,
      "Clause-internal pragmatic marker": 0,
      "Clause-medial pragmatic marker": 24,
      "Discourse marker": 1,
      "Quotative marker": 0,
      "quotative marker": 0,
      "unclassified": 0
    },
    "Discourse marker": {
      "Clause-final pragmatic marker": 0,
      "Clause-initial pragmatic marker": 1,
      "Clause-internal pragmatic marker": 0,
      "Clause-medial pragmatic marker": 18,
      "Discourse marker": 20,
      "Quotative marker": 0,
      "quotative marker": 0,
      "unclassified": 1
    },
    "Quotative marker": {
      "Clause-final pragmatic marker": 0,
      "Clause-initial pragmatic marker": 0,
      "Clause-internal pragmatic marker": 1,
      "Clause-medial pragmatic marker": 2,
      "Discourse marker": 1,
      "Quotative marker": 15,
      "quotative marker": 1,
      "unclassified": 0
    },
    "quotative marker": {
      "Clause-final pragmatic marker": 0,
      "Clause-initial pragmatic marker": 0,
      "Clause-internal pragmatic marker": 0,
      "Clause-medial pragmatic marker": 0,
      "Discourse marker": 0,
      "Quotative marker": 0,
      "quotative marker": 0,
      "unclassified": 0
    },
    "unclassified": {
      "Clause-final pragmatic marker": 0,
      "Clause-initial pragmatic marker": 0,
      "Clause-internal pragmatic marker": 0,
      "Clause-medial pragmatic marker": 5,
      "Discourse marker": 0,
      "Quotative marker": 0,
      "quotative marker": 0,
      "unclassified": 0
    }
  },
  "label_metrics_available": true
}