​
Loops

Loops are the fundamental building blocks of HITL.sh workflows. A loop defines how requests flow from your AI system to human reviewers and back, creating a complete human-in-the-loop cycle.

​
What is a Loop?

A loop in HITL.sh is a configurable workflow that:
  • Receives Requests: Accepts requests from your AI system or applications
  • Routes to Reviewers: Directs requests to appropriate human reviewers
  • Collects Decisions: Gathers human responses and decisions
  • Returns Results: Sends human decisions back to your system
  • Manages Flow: Handles escalations, timeouts, and routing logic

Loop Examples

  • Content moderation workflows
  • Financial approval processes
  • Quality assurance reviews
  • Compliance verification
  • Customer support escalations

​
Loop Components

​
1. Loop Configuration

Every loop has essential configuration settings:

​
2. Request Schema

Define the structure of requests that flow through the loop: 
{
  "content": "string",
  "content_type": "text|image|video",
  "priority": "low|normal|high|urgent",
  "metadata": {
    "user_id": "string",
    "timestamp": "datetime",
    "source": "string"
  },
  "ai_analysis": {
    "confidence": "number",
    "flags": ["array"],
    "risk_score": "number"
  }
}

​
3. Response Options

Configure what decisions reviewers can make:

Approved

Content or request passes review and can proceed.

Rejected

Content or request fails review and is blocked.

Needs Changes

Request modifications before approval.

Escalate

Send to senior reviewer or manager.

​
Loop Lifecycle

​
Creation Phase

1

Define Purpose

Determine what the loop will review and why human oversight is needed.
2

Configure Settings

Set up review requirements, response times, and routing logic.
3

Add Reviewers

Invite team members and assign appropriate skills and permissions.
4

Test Configuration

Verify the loop works correctly with sample requests.

​
Operation Phase

Once active, loops continuously process requests:
Diagram showing the continuous operation of a loop processing requests

​
Maintenance Phase

Regular monitoring and optimization:
  • Performance Review: Analyze response times and decision quality
  • Workload Balancing: Adjust reviewer assignments and skills
  • Process Improvement: Refine routing rules and escalation procedures
  • Team Updates: Add or remove reviewers as needed

​
Loop Types

​
Content Moderation Loops

Review user-generated content for inappropriate material: 
content_moderation_loop = {
    "name": "Content Moderation",
    "description": "Review flagged content for community guidelines",
    "settings": {
        "required_approvals": 1,
        "response_time": 3600,  # 1 hour
        "auto_approve_after": 14400,  # 4 hours
        "escalation_rules": {
            "high_risk": "immediate_escalation",
            "timeout": "senior_reviewer"
        }
    },
    "reviewers": ["moderator_1", "moderator_2", "senior_moderator"],
    "response_options": ["approved", "rejected", "needs_changes", "escalate"]
}

​
Approval Workflow Loops

Handle business process approvals: 
approval_loop = {
    "name": "Expense Approval",
    "description": "Review and approve expense reports",
    "settings": {
        "required_approvals": 2,
        "response_time": 86400,  # 24 hours
        "auto_approve_after": 172800,  # 48 hours
        "routing_rules": {
            "amount_threshold": 1000,
            "escalation": "manager_approval"
        }
    },
    "reviewers": ["supervisor", "manager", "finance_team"],
    "response_options": ["approved", "rejected", "needs_receipts", "escalate"]
}

​
Quality Assurance Loops

Review AI-generated content for accuracy: 
qa_loop = {
    "name": "AI Content QA",
    "description": "Review AI-generated content for quality and accuracy",
    "settings": {
        "required_approvals": 1,
        "response_time": 7200,  # 2 hours
        "auto_approve_after": 28800,  # 8 hours
        "quality_thresholds": {
            "accuracy": 0.95,
            "completeness": 0.90
        }
    },
    "reviewers": ["qa_specialist", "domain_expert", "senior_reviewer"],
    "response_options": ["approved", "rejected", "needs_revision", "escalate"]
}

​
Advanced Loop Features

​
Conditional Routing

Route requests based on content characteristics: 
def route_request(request):
    if request.priority == "urgent":
        return assign_to_senior_reviewer(request)
    elif request.content_type == "image":
        return assign_to_image_specialist(request)
    elif request.ai_confidence < 0.5:
        return assign_to_expert_reviewer(request)
    else:
        return assign_to_available_reviewer(request)

​
Escalation Chains

Handle timeouts and complex decisions: 
escalation_chain = {
    "level_1": {
        "reviewers": ["junior_reviewer"],
        "timeout": 3600,  # 1 hour
        "next_level": "level_2"
    },
    "level_2": {
        "reviewers": ["senior_reviewer"],
        "timeout": 7200,  # 2 hours
        "next_level": "level_3"
    },
    "level_3": {
        "reviewers": ["manager"],
        "timeout": 86400,  # 24 hours
        "final_action": "auto_approve"
    }
}

​
Batch Processing

Group similar requests for efficient review: 
def batch_requests(requests, batch_size=10):
    # Group requests by content type and priority
    batches = {}
    for request in requests:
        key = f"{request.content_type}_{request.priority}"
        if key not in batches:
            batches[key] = []
        batches[key].append(request)
        
        # Send batch when full
        if len(batches[key]) >= batch_size:
            send_batch_for_review(batches[key])
            batches[key] = []

​
Loop Performance Metrics

Track and optimize your loops:

Response Time

  • Average response time
  • 95th percentile response time
  • Time to first response
  • Escalation frequency

Quality Metrics

  • Decision consistency
  • Inter-rater reliability
  • Error rates
  • Reviewer performance

Volume Metrics

  • Requests per day
  • Peak load times
  • Queue length
  • Processing capacity

Cost Metrics

  • Cost per request
  • Reviewer efficiency
  • Automation savings
  • ROI calculations

​
Best Practices

​
Loop Design

1

Start Simple

Begin with basic loops and add complexity as needed.
2

Clear Guidelines

Provide reviewers with explicit decision criteria.
3

Balanced Workload

Distribute work evenly across your reviewer team.
4

Regular Review

Continuously monitor and optimize loop performance.

​
Reviewer Management

  • Skill Matching: Assign reviewers based on expertise
  • Training: Provide regular training on policies and procedures
  • Feedback: Give reviewers feedback on their performance
  • Rotation: Rotate reviewers to prevent burnout and bias

​
Performance Optimization

  • Threshold Tuning: Adjust confidence thresholds based on performance
  • Routing Optimization: Refine routing rules for better efficiency
  • Automation: Automate routine decisions where possible
  • Capacity Planning: Scale reviewer capacity based on demand

​
Next Steps

Ready to create and configure your loops?

Create Your First Loop

Step-by-step guide to building your first loop.

Explore Request Types

Learn about different types of requests your loops can handle.

Mobile App Setup

Get your reviewers set up with the mobile app.