FarmOS Observations

AI-powered quick capture system — field observations, photos, voice notes, and issue reports.

When to Use This

What this skill handles: Field observations -- pest/disease/weed reports, crop condition notes, weather damage, soil issues, equipment problems spotted in the field, and photo-based scouting captures.

Trigger phrases: "found [pest/weed/disease] in field X", "beans look rough", "something is wrong with field 12", "create an observation", "log this problem", "any observations today?", "what has been reported in field X?"

What this does NOT handle: Equipment maintenance scheduling or fleet status (use farmos-equipment), task/work order creation (use farmos-tasks -- but the bot will offer to create a work order after logging an observation), weather forecasts or spray conditions (use farmos-weather).

Minimum viable input: Any mention of something observed in the field. "Beans look bad" is enough -- the bot will ask smart follow-ups.

Data Completeness

1. The /api/integration/dashboard endpoint is for summary stats only — observation counts and pending reviews. Do NOT use it to list individual observations.
2. For listing observations, use GET /api/observations with appropriate filters. This endpoint is paginated — use limit parameter and note the total.
3. Always state the count: "Found 7 observations this week in field 12" — not just a list without context.
4. If results seem low, flag it: "Only seeing 2 observations this week — that may be incomplete, or the observations service may be having issues."
5. If the service is down, say so plainly. Don't present empty results as "no observations."

API Base

http://100.102.77.110:8008

Note: The observations backend may have stability issues (restart loops reported). If endpoints don't respond, report that the observations service appears to be down.

Integration Endpoints (No Auth)

Dashboard

GET /api/integration/dashboard

Returns: Observation counts, recent activity, pending reviews.

Authenticated Endpoints (JWT Required)

Authentication

This skill accesses protected FarmOS endpoints that require a JWT token.

To get a token:

TOKEN=$(~/clawd/scripts/farmos-auth.sh manager)

To use the token:

curl -H "Authorization: Bearer $TOKEN" http://100.102.77.110:8008/api/endpoint

Token expiry: Tokens last 15 minutes. If you get a 401 response, request a new token.

List Observations

GET /api/observations?limit=10&field_id=12 Authorization: Bearer {token}

Observation Detail

GET /api/observations/{id} Authorization: Bearer {token}

Returns: Full observation with AI analysis results, extracted entities, urgency score, and any created actions (tasks, maintenance records).

Create Observation

POST /api/observations Authorization: Bearer {token} Content-Type: multipart/form-data

Form fields:

• observation_type (required) — pest, disease, weed, weather_damage, equipment_issue, soil, crop_condition, other
• description (required) — Text description of what was observed
• severity (optional) — low, medium, high (default: medium)
• field_id (optional) — Numeric field ID
• equipment_id (optional) — Numeric equipment ID
• photo (optional) — Image file attachment

Example using curl:

curl -X POST http://100.102.77.110:8008/api/observations \
  -H "Authorization: Bearer $TOKEN" \
  -F "observation_type=weed" \
  -F "description=Found waterhemp in northeast corner near waterway" \
  -F "severity=high" \
  -F "field_id=22" \
  -F "photo=@/path/to/photo.jpg"

When crew reports a problem in #field-support or #field-ops, offer to create an observation. Extract as much detail as you can from the message (field, observation type, severity), then create the observation.

Usage Notes

• Observations include urgency scores (1-10). Flag anything 7+ immediately.
• AI processing classifies type and extracts equipment/field/crop references.
• Observations may create tasks (via Task Manager) or maintenance records (via Equipment).
• If the service is down, let the user know and suggest they log the observation manually.
• The observation intake system is designed for photos from the field — the bot should be ready to accept image messages and route them here.
• Proactive observation creation: When crew mentions issues in channel conversations ("got a bunch of weeds in field 12", "header making a weird noise"), offer to log it as an observation. Don't create silently — ask first.
• Equipment observations: If the observation involves equipment, include equipment_id when creating. This helps track equipment-specific recurring issues.

Smart Observation Detection

When a user reports something that sounds like a field observation, auto-detect as much as you can from the message before asking questions.

What to Detect

Field identification:

• Explicit: "field 12", "F12", "the 12"
• By name: "the Byrd farm", "Kruckeberg", "home place" -- match to known field names
• From channel context: if the conversation was already about a specific field, carry that forward
• From user location: if they mention "the field I am in" or "out here", check recent context

Observation type -- see the Observation Type Detection table below.

Severity -- see the Severity Detection table below.

Specific pest/disease/weed identification:

• Common Indiana pests: western corn rootworm, Japanese beetle, corn earworm, soybean aphid, bean leaf beetle, armyworm, black cutworm, stink bug
• Common diseases: tar spot, gray leaf spot, northern leaf blight, sudden death syndrome, white mold, frogeye leaf spot, Goss wilt, anthracnose
• Common weeds: waterhemp, marestail (horseweed), giant ragweed, common ragweed, Palmer amaranth, lambsquarters, foxtail, velvetleaf, morningglory
• If the reporter uses a colloquial name, map it: "buttonweed" -> common buttonweed, "volunteer corn" -> note as weed/volunteer

Equipment reference:

• By name/number: "the 8250", "the Kinze", "the planter", "sprayer"
• By implication: "header won't raise" implies the combine (likely 8250)

Location within field:

• Cardinal directions: "northeast corner", "south end"
• Landmarks: "near the waterway", "along the tree line", "by the road", "headlands", "terrace"
• Coverage: "whole field", "scattered", "in patches", "one spot"

Observation Type Detection

If multiple types match (e.g., "yellowing leaves with spots" could be disease or crop_condition), pick the more specific one (disease in that case). If genuinely ambiguous, ask: "Is this more of a disease issue or general crop condition?"

Severity Detection

Default to medium if the language is neutral or you cannot determine severity. Never guess high -- ask.

Follow-Up Questions (Smart)

When the report is sparse, ask targeted follow-up questions. Maximum 2-3 questions per interaction -- do not interrogate.

Question Bank (pick the most useful ones)

• Scope: "How widespread is it? Just that spot or across the field?"
• Adjacent fields: "Have you seen this in adjacent fields?"
• Percentage: "Roughly what percentage of the field is affected?"
• Photos: "Any photos? A picture would help me nail down exactly what this is."
• Treatment history: "Was this field treated with anything recently?"
• Recurrence: "Is this the same issue you reported on [date]?" (only if a similar recent observation exists -- check observation history first)
• Timing: "When did you first notice this?"
• Trend: "Is it getting worse or holding steady?"

Which Questions to Ask

Reporter-Adaptive Behavior

Detailed Reporters (agronomists, Brian, experienced crew)

• They will say things like "Found western corn rootworm beetle damage in field 22, northeast corner, about 15% of the rows affected, adult count was 8 per plant"
• Auto-detect everything. Confirm the details. Create the observation.
• Skip redundant questions -- they have already given you what you need
• Example response: "Got it -- rootworm beetle damage in F22, northeast corner, ~15% of rows, 8 adults/plant. Severity: high. Creating the observation now."

Sparse Reporters (field crew, seasonal workers)

• They will say things like "something's wrong with field 12" or "beans look bad"
• Ask 2-3 targeted follow-up questions. Do NOT interrogate.
• Pick the highest-value questions for the situation
• After 2-3 questions, create the observation with whatever you have
• Example exchange:
  • Crew: "Beans look rough in 12"
  • Bot: "What are you seeing -- weeds, bugs, disease, or just the crop looking off? And is it the whole field or just a section?"
  • Crew: "Weeds. Northeast side mostly."
  • Bot: "Got it. Creating an observation -- weed pressure in field 12, northeast section, medium severity. Want me to create a scouting work order for it too?"

Key Rules

• Always offer before creating: "Want me to create an observation for this?" Do NOT create silently -- unless the user has explicitly asked you to create one.
• Exception: If the user directly says "Log this" or "Create an observation for...", that IS explicit permission -- go ahead and create.
• Never ask more than 3 follow-up questions. After 3, create with what you have and note what was unknown.

Post-Creation Actions

After successfully creating an observation, offer related actions:

• Work order: "Want me to create a work order for this?" (especially for weed/pest/equipment issues that need action)
• Adjacent field check: "Should I check if adjacent fields have the same issue?" (useful for pest/disease/weed spread)
• Scouting task: "Want me to schedule a follow-up scouting trip?" (for observations that need monitoring)
• Equipment maintenance: If equipment_issue type, "Want me to log this against the equipment maintenance record?"

Only offer 1-2 of the most relevant follow-up actions. Do not overwhelm the reporter with options.

Urgency Escalation

Flag the operator immediately (in addition to creating the observation) for:

• Crop damage at high severity -- potential yield loss, Brian needs to know
• Equipment safety -- hydraulic leak, structural failure, anything that could injure someone
• Chemical exposure -- drift, spill, re-entry violation, any chemical safety concern
• Pest/disease outbreak -- high severity pest or disease that could spread rapidly (tar spot, sudden death syndrome, heavy rootworm pressure)
• Weather damage at high severity -- hail, flood, significant storm damage

When escalating, send a concise alert: "URGENT: [type] reported in field [X] -- [one-line summary]. Severity: high. Observation #[id] created."

Do NOT escalate routine observations (low/medium severity, isolated issues, normal scouting finds).

Cross-Module Context

After creating or reviewing observations, connect to other modules:

Observations → Pattern Detection:

• When a new observation is created, check for similar recent observations (same type, nearby fields, same week): "This is the third waterhemp observation this week across three different fields. Might be time for a blanket spray program rather than spot treatments."
• When listing observations, group by pattern when multiple share type/field/timeframe: "3 disease observations this week, all in the east fields. Could be spreading."
• Track escalation: "First report was low severity last Tuesday, now we're at high severity across 4 fields. This is moving fast."

Observations → Tasks:

• After creating an observation, check farmos-tasks for existing work orders related to this field or issue: "There's already a scouting task open for field 22 from yesterday — want me to add this observation to it?"
• If no related task exists and the observation is actionable (pest, disease, weed at medium+ severity), offer to create one.
• Connect observation patterns to task suggestions: "Third waterhemp sighting this week — want me to create a blanket spray task?"

Observations → Weather:

• Connect recent weather to observation context: "3 disease observations after last week's rain — moisture likely drove this."
• When a weather_damage observation is created, pull the actual weather data: "You're reporting hail damage in field 14. Records show we got 1.2 inches with possible hail Tuesday evening."
• Flag ongoing weather risk: "With more rain coming Thursday, expect this fungal pressure to continue."

Observations → Equipment:

• If an observation leads to a task that requires specific equipment, check equipment availability: "If you're going to spray for this waterhemp, the sprayer is available — 153 hours, no maintenance due."
• For equipment_issue observations, cross-reference with farmos-equipment for that machine's maintenance history.

Cross-reference when it adds context. A simple "log this observation" doesn't need a full cross-module sweep. But when patterns emerge or observations drive action, connect the dots.

Image Understanding

When a photo accompanies an observation (the image description will appear in your context as [Image] Description: ...), use it to enhance the observation record.

Photo-Enhanced Detection

Use the image description to refine type and severity:

• Photo of lesions on leaves -- identify disease characteristics (shape, color, pattern, location on leaf) -- refine observation_type to "disease" and attempt specific ID
• Photo of insects or insect damage -- identify species or damage pattern -- refine observation_type to "pest"
• Photo of weeds -- identify species from leaf shape, growth habit, flower/seed head -- refine observation_type to "weed"
• Photo of equipment in the field -- note the machine and any visible issues -- set observation_type to "equipment_issue" and include equipment_id
• Photo of crop conditions -- note growth stage, color, stand count, uniformity -- refine observation_type to "crop_condition"
• Photo of weather damage -- note damage pattern (hail bruising, wind lodging, flood line) -- set observation_type to "weather_damage"

Always include the image description in the observation description field. Combine what the reporter said with what the photo shows:

"Reporter: Found some weird spots on the corn in field 12. Photo shows: rectangular tan lesions between leaf veins, approximately 1-3cm long, consistent with gray leaf spot (Cercospora zeae-maydis). Northeast section of field."

Photo Quality Handling

• Clear photo: Use it confidently to refine detection. State what you see and your assessment.
• Unclear/blurry/dark photo: Say so honestly: "I can make out [what you can see] but the photo is too blurry/dark for a confident ID. Can you get a closer shot, or describe what you are seeing?"
• Photo does not match description: If the photo shows something different from what the reporter described, mention it: "You mentioned weeds but the photo looks like it might be disease lesions -- can you clarify?"
• Multiple issues visible: Note all of them: "I can see both waterhemp and what looks like tar spot lesions in this photo. Want me to create observations for both?"

Photo Prompt for Sparse Reports

When a reporter sends a text-only observation about something visual (pest, disease, weed, damage), and has NOT included a photo:

• "Any photos? A picture would help me nail down exactly what this is." (already in the follow-up question bank)
• Do NOT demand photos. A text description is always enough to create an observation.

Attaching Photos to Observations

When creating an observation via POST /api/observations, include the photo as the photo form field if a MediaPath is available in your context. The image gets archived with the observation record for future reference.