Loops

Loops are the foundational organizational structure in HITL.sh that connect human reviewers with content and decisions requiring oversight. Think of a loop as a team or group of reviewers who collaborate to handle specific types of requests from your applications.

What is a Loop?

A loop in HITL.sh is a structured group that:

Manages Reviewers: Organizes human reviewers who can respond to requests
Receives Requests: Accepts requests created via the API for human review
Broadcasts Notifications: Sends push notifications to mobile devices when new requests arrive
Tracks Member Activity: Monitors who’s active and available to respond
Controls Access: Manages who can create requests and who can review them

Loops are created and managed by API key holders, while reviewers join loops via the mobile app using invite codes or QR codes.

Loop Components

Basic Properties

Every loop has fundamental properties that define its identity and behavior:

Name & Description

Name: Clear, descriptive identifier for the loop (required, max 100 characters)
Description: Detailed explanation of the loop’s purpose and scope (optional, max 500 characters)
Icon: Visual identifier for the loop (required, max 100 characters - typically an emoji or short text)
Creator: User who created and owns the loop

Membership

Active Members: Reviewers who can receive and respond to requests
Pending Members: Users who’ve been invited but haven’t joined yet
Member Count: Total number of people in the loop
Invite Code: Unique code for joining the loop

Access Control

Creator Permissions: Only loop creators can modify settings and add requests
Member Permissions: Members can only view and respond to requests
API Access: Only the creator’s API key can create requests in the loop

Invite System

Loops use a simple but effective invite system:

# Create a loop and get invite details
import requests

loop_data = {
    "name": "Content Moderation Team",
    "description": "Reviews user-generated content for community guidelines compliance",
    "icon": "🛡️"
}

response = requests.post(
    "https://api.hitl.sh/v1/api/loops",
    headers={"Authorization": f"Bearer {api_key}"},
    json=loop_data
)

loop_response = response.json()["data"]
loop = loop_response["loop"]
print(f"Loop created: {loop['name']}")
print(f"Invite code: {loop_response['invite_code']}")
print(f"QR code URL: {loop_response['qr_code_url']}")
print(f"Join URL: {loop_response['join_url']}")

Reviewers can join using:

QR Code: Scan with the HITL mobile app camera
Invite Code: Enter the 6-digit code in the app
Direct Link: Share URL that opens the mobile app directly

Loop Lifecycle

1. Creation

Loops are created via the API by users who need human reviewers:

import requests

# Create a new loop
loop_data = {
    "name": "Financial Transaction Review",
    "description": "Human oversight for suspicious financial transactions",
    "icon": "💰"
}

response = requests.post(
    "https://api.hitl.sh/v1/api/loops",
    headers={
        "Authorization": f"Bearer {api_key}",
        "Content-Type": "application/json"
    },
    json=loop_data
)

if response.status_code == 201:
    loop = response.json()["data"]["loop"]
    print(f"✅ Loop created: {loop['id']}")
    print(f"📱 Share invite code: {loop['invite_code']}")
else:
    print("❌ Failed to create loop:", response.json())

const axios = require('axios');

async function createLoop() {
    const loopData = {
        name: "Product Review Team",
        description: "Reviews product listings and descriptions for accuracy",
        icon: "📦"
    };

    try {
        const response = await axios.post(
            "https://api.hitl.sh/v1/api/loops",
            loopData,
            {
                headers: {
                    "Authorization": `Bearer ${apiKey}`,
                    "Content-Type": "application/json"
                }
            }
        );

        const loop = response.data.data.loop;
        console.log(`✅ Loop created: ${loop.name}`);
        console.log(`📱 Invite code: ${loop.invite_code}`);
        return loop;
    } catch (error) {
        console.error("❌ Failed to create loop:", error.response?.data);
    }
}

curl -X POST https://api.hitl.sh/v1/api/loops \
  -H "Authorization: Bearer your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Customer Support Escalation",
    "description": "Human agents review escalated customer support tickets",
    "icon": "🎧"
  }'

2. Reviewer Recruitment

Once created, loops need human reviewers to function:

Share Invite Code

Distribute the 6-digit invite code to potential reviewers via email, Slack, or other communication channels.

QR Code Sharing

Use the provided QR code URL to generate scannable codes for easy mobile app joining.

Direct Mobile Links

Create deep links that open the HITL mobile app directly to the join screen.

Monitor Membership

Track who joins and ensure you have adequate reviewer coverage.

3. Active Operation

With reviewers in place, loops continuously process requests:

# Monitor loop activity
response = requests.get(
    f"https://api.hitl.sh/v1/api/loops/{loop_id}",
    headers={"Authorization": f"Bearer {api_key}"}
)

loop = response.json()["data"]["loop"]
print(f"Loop: {loop['name']}")
print(f"Active members: {loop['member_count'] - loop['pending_count']}")
print(f"Pending invites: {loop['pending_count']}")

# Get recent requests for this loop
requests_response = requests.get(
    f"https://api.hitl.sh/v1/api/loops/{loop_id}/requests",
    headers={"Authorization": f"Bearer {api_key}"}
)

requests_data = requests_response.json()["data"]["requests"]
print(f"Total requests: {len(requests_data)}")

# Count by status
pending = len([r for r in requests_data if r["status"] == "pending"])
completed = len([r for r in requests_data if r["status"] == "completed"])
print(f"Pending: {pending}, Completed: {completed}")

4. Management & Updates

Loop creators can modify settings and membership:

# Update loop information
update_data = {
    "name": "Enhanced Content Moderation",
    "description": "Advanced review team with updated guidelines and training",
    "icon": "🔍"
}

response = requests.put(
    f"https://api.hitl.sh/v1/api/loops/{loop_id}",
    headers={"Authorization": f"Bearer {api_key}"},
    json=update_data
)

if response.status_code == 200:
    print("✅ Loop updated successfully")
else:
    print("❌ Update failed:", response.json())

# Remove inactive members
member_id = "65f1234567890abcdef12346"

response = requests.delete(
    f"https://api.hitl.sh/v1/api/loops/{loop_id}/members/{member_id}",
    headers={"Authorization": f"Bearer {api_key}"}
)

if response.status_code == 200:
    print("✅ Member removed successfully")
else:
    print("❌ Removal failed:", response.json())

Loop Types & Use Cases

Content Moderation Loops

Review user-generated content for policy compliance:

moderation_loop = {
    "name": "Community Content Review",
    "description": "Reviews posts, comments, and user uploads for community guideline violations",
    "icon": "🛡️"
}

# Typical requests for this loop type
request_data = {
    "processing_type": "time-sensitive",
    "type": "markdown",
    "priority": "high",
    "request_text": f"Review this comment: '{user_comment}'",
    "response_type": "single_select",
    "response_config": {
        "options": [
            {"value": "approve", "label": "✅ Approve", "color": "#22c55e"},
            {"value": "reject", "label": "❌ Reject", "color": "#ef4444"},
            {"value": "warn", "label": "⚠️ Warn User", "color": "#f59e0b"}
        ],
        "required": True
    },
    "default_response": "reject",
    "timeout_seconds": 1800,  # 30 minutes
    "platform": "api"
}

Quality Assurance Loops

Review AI-generated or automated content:

qa_loop = {
    "name": "AI Content Quality Review",
    "description": "Human experts review AI-generated content for accuracy and quality",
    "icon": "🎯"
}

# Quality rating request
request_data = {
    "processing_type": "deferred",
    "type": "markdown", 
    "priority": "medium",
    "request_text": f"Rate the quality of this AI-generated article:\n\n{article_content}",
    "response_type": "rating",
    "response_config": {
        "scale_min": 1,
        "scale_max": 10,
        "scale_step": 0.5,
        "labels": {
            "1": "Poor - Needs complete rewrite",
            "5": "Average - Acceptable with minor edits", 
            "10": "Excellent - Publish as-is"
        },
        "required": True
    },
    "default_response": 5,
    "timeout_seconds": 86400,  # 24 hours
    "platform": "api"
}

Business Process Loops

Handle approval workflows and decision-making:

approval_loop = {
    "name": "Expense Approval Team",
    "description": "Reviews and approves employee expense reports and reimbursements",
    "icon": "💼"
}

# Approval decision request
request_data = {
    "processing_type": "time-sensitive",
    "type": "markdown",
    "priority": "medium",
    "request_text": f"Approve this expense report:\n\nAmount: ${amount}\nCategory: {category}\nDescription: {description}",
    "response_type": "single_select",
    "response_config": {
        "options": [
            {"value": "approve", "label": "✅ Approve Expense"},
            {"value": "reject", "label": "❌ Reject Expense"}
        ],
        "required": True
    },
    "default_response": "reject",  # Conservative default
    "timeout_seconds": 14400,  # 4 hours
    "platform": "api"
}

Verification & Compliance Loops

Verify information accuracy and regulatory compliance:

compliance_loop = {
    "name": "Data Verification Team",
    "description": "Verifies business information and ensures regulatory compliance",
    "icon": "📋"
}

# Multi-select verification request
request_data = {
    "processing_type": "deferred",
    "type": "markdown",
    "priority": "low", 
    "request_text": f"Verify this business listing:\n\n{business_info}",
    "response_type": "multi_select",
    "response_config": {
        "options": [
            {"value": "address_verified", "label": "📍 Address Verified"},
            {"value": "phone_verified", "label": "📞 Phone Verified"},
            {"value": "hours_verified", "label": "🕒 Hours Verified"},
            {"value": "website_verified", "label": "🌐 Website Verified"},
            {"value": "license_verified", "label": "📜 License Verified"}
        ],
        "min_selections": 1,
        "max_selections": 5,
        "required": True
    },
    "default_response": "",
    "timeout_seconds": 259200,  # 3 days
    "platform": "api"
}

Loop Management

Monitoring Loop Health

Keep track of loop performance and member engagement:

def analyze_loop_performance(loop_id):
    # Get loop details
    loop_response = requests.get(
        f"https://api.hitl.sh/v1/api/loops/{loop_id}",
        headers={"Authorization": f"Bearer {api_key}"}
    )
    loop = loop_response.json()["data"]["loop"]
    
    # Get recent requests
    requests_response = requests.get(
        f"https://api.hitl.sh/v1/api/loops/{loop_id}/requests",
        headers={"Authorization": f"Bearer {api_key}"}
    )
    requests_data = requests_response.json()["data"]["requests"]
    
    # Analyze metrics
    total_requests = len(requests_data)
    completed_requests = [r for r in requests_data if r["status"] == "completed"]
    timeout_requests = [r for r in requests_data if r["status"] == "timeout"]
    
    completion_rate = len(completed_requests) / total_requests if total_requests > 0 else 0
    timeout_rate = len(timeout_requests) / total_requests if total_requests > 0 else 0
    
    print(f"Loop: {loop['name']}")
    print(f"Members: {loop['member_count']} ({loop['member_count'] - loop['pending_count']} active)")
    print(f"Requests: {total_requests} total, {len(completed_requests)} completed")
    print(f"Completion rate: {completion_rate:.1%}")
    print(f"Timeout rate: {timeout_rate:.1%}")
    
    # Performance insights
    if completion_rate < 0.8:
        print("⚠️ Low completion rate - consider adding more reviewers")
    if timeout_rate > 0.2:
        print("⚠️ High timeout rate - consider adjusting timeouts or priorities")
    if loop['member_count'] - loop['pending_count'] < 3:
        print("⚠️ Few active members - recruit more reviewers for reliability")

# Monitor your loops
analyze_loop_performance(loop_id)

Member Management

Track and manage loop membership:

# Get detailed member information
members_response = requests.get(
    f"https://api.hitl.sh/v1/api/loops/{loop_id}/members",
    headers={"Authorization": f"Bearer {api_key}"}
)

members = members_response.json()["data"]["members"]

print("Loop Members:")
for member in members:
    status_emoji = "✅" if member["status"] == "active" else "⏳"
    joined_date = member.get("joined_at", "Unknown")
    print(f"  {status_emoji} {member['name']} ({member['email']}) - Joined: {joined_date}")

# Remove inactive members if needed
inactive_members = [m for m in members if m["status"] == "pending" and is_old_invite(m)]
for member in inactive_members:
    print(f"Removing inactive member: {member['email']}")
    requests.delete(
        f"https://api.hitl.sh/v1/api/loops/{loop_id}/members/{member['user_id']}",
        headers={"Authorization": f"Bearer {api_key}"}
    )

Best Practices

Loop Design

Clear Purpose

Give loops specific, well-defined purposes rather than general “review everything” mandates.

Right Size

Aim for 3-10 active members per loop - enough for coverage but small enough for accountability.

Descriptive Names

Use names that clearly indicate what the loop reviews and how urgent it is.

Regular Maintenance

Periodically review membership, remove inactive users, and update descriptions.

Operational Efficiency

Redundancy Planning

Always have multiple active reviewers to ensure coverage during off-hours or vacations.

Priority Management

Use different loops for different priority levels rather than mixing urgent and routine requests.

Specialization

Create specialized loops for different types of content (text, images, different domains).

Performance Monitoring

Regularly check completion rates and response times to identify bottlenecks.

Reviewer Experience

Clear Guidelines: Provide reviewers with explicit instructions and examples
Reasonable Workload: Don’t overwhelm loops with too many requests
Feedback Loop: Use the feedback API to acknowledge good reviewer performance
Training Materials: Share documentation about what to look for and how to respond

Scaling Patterns

Single Loop → Multiple Loops

As your needs grow, consider splitting loops:

# Start with one general loop
general_loop = create_loop("Content Review", "Reviews all user content")

# Split into specialized loops as volume increases
text_loop = create_loop("Text Content Review", "Reviews posts, comments, and messages")
image_loop = create_loop("Image Content Review", "Reviews uploaded photos and graphics") 
urgent_loop = create_loop("Urgent Review Team", "High-priority content requiring immediate attention")

Geographic Distribution

Create region-specific loops for timezone coverage:

# Timezone-based loops for 24/7 coverage
us_loop = create_loop("US Review Team", "Primary coverage: 9 AM - 5 PM EST")
eu_loop = create_loop("EU Review Team", "Primary coverage: 9 AM - 5 PM CET")  
asia_loop = create_loop("Asia Review Team", "Primary coverage: 9 AM - 5 PM JST")

# Route requests based on urgency and time
def route_request_by_timezone():
    current_hour_utc = datetime.utcnow().hour
    
    if 13 <= current_hour_utc <= 21:  # US business hours
        return us_loop["id"]
    elif 8 <= current_hour_utc <= 16:   # EU business hours
        return eu_loop["id"] 
    else:                               # Asia business hours
        return asia_loop["id"]

Hierarchical Review

Set up escalation between loops:

# Primary review loop
primary_loop = create_loop("First-Level Review", "Initial content screening")

# Escalation loop for complex cases
escalation_loop = create_loop("Senior Review Team", "Handles escalated and complex cases")

# In your request handling:
if reviewer_selected == "escalate":
    # Create new request in escalation loop
    escalated_request = create_request(
        loop_id=escalation_loop["id"],
        request_text=f"Escalated from primary review:\n\n{original_request}",
        # ... other configuration
    )

Troubleshooting

Common Issues

No Active Members

Problem: Requests timeout because no reviewers are available.Solutions:

Check that invite codes have been shared and used
Verify reviewers have the mobile app installed and notifications enabled
Consider timezone differences when recruiting reviewers
Monitor member activity and remove/replace inactive users

Low Response Rates

Problem: Many requests timeout instead of being completed.Solutions:

Increase timeout durations to allow more response time
Add more reviewers to increase coverage
Adjust priorities - too many high-priority requests reduce effectiveness
Check if request instructions are clear and actionable

Inconsistent Responses

Problem: Reviewers give different answers to similar requests.Solutions:

Provide clearer guidelines and examples
Use more structured response types (single select vs. text)
Consider reviewer training or calibration exercises
Review response configurations to ensure options are clear

Member Management Issues

Problem: Can’t remove members or invite codes aren’t working.Solutions:

Only loop creators can manage membership - including removing members, updating loop details, and deleting loops
Loop creators cannot remove themselves from the loop
Invite codes are case-sensitive and expire after extended periods
Removed members can rejoin with the same invite code if needed
Check API responses for specific error messages

Next Steps

Create Your First Loop

Start building your reviewer team with the loops API

Request Management

Learn how to create and manage requests within your loops

Mobile App Setup

Help your reviewers get set up with the HITL mobile app

API Reference

Explore all available loop management endpoints

​Loops

​What is a Loop?

​Loop Components

​Basic Properties

​Invite System

​Loop Lifecycle

​1. Creation

​2. Reviewer Recruitment

​3. Active Operation

​4. Management & Updates

​Loop Types & Use Cases

​Content Moderation Loops

​Quality Assurance Loops

​Business Process Loops

​Verification & Compliance Loops

​Loop Management

​Monitoring Loop Health

​Member Management

​Best Practices

​Loop Design

​Operational Efficiency

Redundancy Planning

Priority Management

Specialization

Performance Monitoring

​Reviewer Experience

​Scaling Patterns

​Single Loop → Multiple Loops

​Geographic Distribution

​Hierarchical Review

​Troubleshooting

​Common Issues

​Next Steps

Create Your First Loop

Request Management

Mobile App Setup

API Reference

Loops

What is a Loop?

Loop Components

Basic Properties

Invite System

Loop Lifecycle

1. Creation

2. Reviewer Recruitment

3. Active Operation

4. Management & Updates

Loop Types & Use Cases

Content Moderation Loops

Quality Assurance Loops

Business Process Loops

Verification & Compliance Loops

Loop Management

Monitoring Loop Health

Member Management

Best Practices

Loop Design

Operational Efficiency

Reviewer Experience

Scaling Patterns

Single Loop → Multiple Loops

Geographic Distribution

Hierarchical Review

Troubleshooting

Common Issues

Next Steps