Skip to main content

Creating Your First Loop

A loop in HITL.sh is a human-in-the-loop workflow that routes requests to human reviewers when AI systems need human oversight. This guide walks you through creating your first loop, adding reviewers, and processing your first request.

What You’ll Build

In this tutorial, you’ll create a Content Moderation Loop that:
  • Receives flagged content from your application
  • Automatically invites reviewers with QR codes and invite links
  • Routes requests to human reviewers via mobile notifications
  • Returns structured human decisions back to your system
By the end of this guide, you’ll have a fully functional loop with invite codes, QR codes, and the ability to process real requests.

Prerequisites

Before you begin, ensure you have:
  • ✅ A HITL.sh account (sign up at my.hitl.sh)
  • ✅ Your API key generated and ready
  • ✅ Basic understanding of REST APIs
  • ✅ Team members who will act as reviewers (with email addresses)

Generate API Key

If you haven’t created your API key yet, complete this step first.

Step 1: Create the Loop

Using the API

Create your loop using the HITL.sh API. When you create a loop, you automatically become a member and receive invitation codes for sharing:
curl -X POST 'https://api.hitl.sh/v1/api/loops' \
  -H 'Authorization: Bearer YOUR_API_KEY' \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "Content Moderation Review",
    "description": "Review user-generated content for community guidelines compliance",
    "icon": "shield-check"
  }'
Expected Response:
{
  "error": false,
  "msg": "Loop created successfully with QR code",
  "data": {
    "loop": {
      "id": "65f1234567890abcdef12345",
      "name": "Content Moderation Review",
      "description": "Review user-generated content for community guidelines compliance",
      "icon": "shield-check",
      "creator_id": "65f1234567890abcdef12346",
      "member_count": 1,
      "pending_count": 0,
      "created_at": "2024-03-15T10:30:00Z"
    },
    "invite_code": "ABC123DEF",
    "qr_code_base64": "data:image/png;base64,iVBORw0KGg...",
    "qr_code_url": "https://api.hitl.sh/qr/ABC123DEF.png", 
    "join_url": "https://my.hitl.sh/join/ABC123DEF"
  }
}
Save the invite_code, join_url, and qr_code_url from the response - you’ll need these to invite reviewers to your loop.

Step 2: Add Reviewers

Your loop needs human reviewers to process requests. Share the invitation information from Step 1 to add team members:

Invitation Methods

Share Invite Code

Give reviewers the invite code (e.g., ABC123DEF) to enter in the mobile app.

Share Join URL

Send the join URL via email or chat for one-click joining.

Share QR Code

Display or share the QR code image for quick mobile scanning.

Email Invitation

Send invitation emails directly through your dashboard.

Mobile App Setup

Reviewers need the HITL.sh mobile app to receive notifications and respond to requests:
1

Download App

Reviewers download the HITL.sh mobile app from their app store.
2

Join Loop

Using one of the invitation methods above (code, URL, or QR scan).
3

Enable Notifications

Ensure push notifications are enabled for instant request alerts.
4

Test Connection

Reviewers should see the loop appear in their app and receive a welcome notification.

Mobile App Guide

Step-by-step guide for reviewers to join loops using the mobile app.

Step 3: Create Your First Request

Now that your loop is set up with reviewers, create your first request to test the workflow:

Request Structure

Requests in HITL.sh consist of:
  • processing_type: time-sensitive or deferred
  • type: Content type (markdown or image)
  • priority: low, medium, high, or critical
  • request_text: The main content to review (1-2000 characters)
  • response_type: Expected response format
  • response_config: Configuration for the response type
  • default_response: Fallback response if timeout occurs
  • platform: Source platform creating the request
  • image_url: Required when type is image
  • context: Additional JSON data for context
  • timeout_seconds: Custom timeout (60-86400 seconds)
  • callback_url: Webhook URL for response notifications
  • platform_version: Version of the creating platform

Response Type Options

Configure what decisions reviewers can make:

Single Select

Choose one option (Approve, Reject, Escalate)

Multi Select

Select multiple issues or categories

Text

Provide detailed written feedback

Rating

Rate content on a numeric scale

Number

Enter specific numeric values

Boolean

Simple yes/no or true/false decisions

Step 4: Submit Your First Request

Create a test request to verify your loop is working properly. Use the loop ID from Step 1:
curl -X POST 'https://api.hitl.sh/v1/api/loops/YOUR_LOOP_ID/requests' \
  -H 'Authorization: Bearer YOUR_API_KEY' \
  -H 'Content-Type: application/json' \
  -d '{
    "processing_type": "time-sensitive",
    "type": "markdown",
    "priority": "medium",
    "request_text": "Please review this user comment: \"This post is amazing! Thanks for sharing.\"",
    "context": {
      "user_id": "user123", 
      "post_id": "post456",
      "automated_flags": ["potential_spam"]
    },
    "timeout_seconds": 3600,
    "response_type": "single_select",
    "response_config": {
      "options": ["Approve", "Reject", "Needs Changes", "Escalate"]
    },
    "default_response": "Approve",
    "platform": "api",
    "callback_url": "https://your-app.com/webhook/response"
  }'
Expected Response:
{
  "error": false,
  "msg": "Request created and broadcasted successfully",
  "data": {
    "request_id": "65f1234567890abcdef12348",
    "status": "pending",
    "processing_type": "time-sensitive",
    "type": "markdown", 
    "priority": "medium",
    "timeout_at": "2024-03-15T11:30:00Z",
    "broadcasted_to": 4,
    "notifications_sent": 3,
    "polling_url": "/v1/api/requests/65f1234567890abcdef12348"
  }
}
Your reviewers will immediately receive push notifications on their mobile devices about this new request!

Step 5: Monitor Request Progress

Track your request and wait for reviewer responses:

Check Request Status

Use the polling URL from the response to check status:
import requests
import time

def wait_for_response(request_id, api_key, max_wait=300):
    """Poll for request completion"""
    url = f"https://api.hitl.sh/v1/api/requests/{request_id}"
    headers = {"Authorization": f"Bearer {api_key}"}
    
    start_time = time.time()
    while time.time() - start_time < max_wait:
        response = requests.get(url, headers=headers)
        data = response.json()
        
        status = data['data']['status']
        print(f"📊 Request status: {status}")
        
        if status == 'completed':
            response_data = data['data']['response_data'] 
            print(f"✅ Human decision: {response_data}")
            return response_data
        elif status == 'cancelled':
            print("❌ Request was cancelled")
            return None
        elif status == 'timed_out':
            print("⏰ Request timed out, using default response")
            return data['data']['default_response']
            
        time.sleep(10)  # Wait 10 seconds before checking again
    
    print("⏰ Polling timeout reached")
    return None

# Check your request
response = wait_for_response("YOUR_REQUEST_ID", "YOUR_API_KEY")

Request Lifecycle

1

Created & Broadcasted

Request is created and push notifications sent to all active reviewers.
2

Pending Review

Waiting for a reviewer to claim and respond to the request.
3

Claimed

A reviewer has opened the request and is working on it.
4

Completed

Reviewer has submitted their response - human decision is ready!
5

Webhook Notification

If configured, your callback URL receives the response data.

Receiving Human Decisions

Set up webhooks for real-time notifications when requests complete:
# Your webhook endpoint
@app.route('/webhook/hitl', methods=['POST'])
def handle_hitl_webhook():
    payload = request.json
    
    if payload.get('event') == 'request.completed':
        request_id = payload['data']['request_id']
        response_data = payload['data']['response_data']
        
        print(f"📥 Received response for {request_id}: {response_data}")
        
        # Process the human decision
        if response_data == 'Approve':
            approve_content(request_id)
        elif response_data == 'Reject':
            reject_content(request_id) 
        elif response_data == 'Escalate':
            escalate_to_manager(request_id)
    
    return {'status': 'received'}, 200

Using Polling

If webhooks aren’t available, poll the request status periodically:
async function pollForResponse(requestId) {
    const maxAttempts = 30; // 5 minutes with 10s intervals
    
    for (let i = 0; i < maxAttempts; i++) {
        try {
            const response = await axios.get(
                `https://api.hitl.sh/v1/api/requests/${requestId}`,
                { headers: { 'Authorization': `Bearer ${apiKey}` }}
            );
            
            const { status, response_data } = response.data.data;
            
            if (status === 'completed') {
                console.log('✅ Human decision received:', response_data);
                return response_data;
            } else if (status === 'timed_out') {
                console.log('⏰ Request timed out');
                return null;
            }
            
            // Wait 10 seconds before next poll
            await new Promise(resolve => setTimeout(resolve, 10000));
            
        } catch (error) {
            console.error('Error polling request:', error);
            break;
        }
    }
    
    console.log('⏰ Polling timeout reached');
    return null;
}

Best Practices

Clear Request Context

Provide sufficient context in request_text and context fields for informed decisions.

Appropriate Timeouts

Set realistic timeout_seconds based on request complexity and reviewer availability.

Meaningful Default Responses

Choose safe default responses that represent the best fallback decision.

Response Type Selection

Match response types to your use case - single select for decisions, text for feedback.

Request Optimization

  • Critical: Security issues, policy violations (< 15 min response)
  • High: Time-sensitive content decisions (< 1 hour)
  • Medium: Standard content review (< 4 hours)
  • Low: Quality improvements, feedback (< 24 hours)
Include relevant context to help reviewers make informed decisions:
  • User information and history
  • Automated system flags and confidence scores
  • Related content or previous decisions
  • Business context and implications
Design response options that are:
  • Mutually exclusive for single select
  • Comprehensive covering all possible decisions
  • Clear and actionable with no ambiguity
  • Consistent with your business logic

Troubleshooting

Symptoms: notifications_sent: 0 in responseSolutions:
  • Verify reviewers have joined the loop successfully
  • Check if reviewers have push notifications enabled
  • Ensure reviewers are active (not in do-not-disturb mode)
  • Confirm the mobile app is installed and logged in
Symptoms: Request created but reviewers don’t see itSolutions:
  • Check if broadcasted_to count matches expected reviewers
  • Verify reviewers are members of the correct loop
  • Ensure loop ID in the request URL is correct
  • Check if reviewers are filtering requests by priority
Symptoms: Request status always shows “pending”Solutions:
  • Verify the request ID is correct
  • Check API key permissions for request access
  • Ensure you’re polling the correct endpoint
  • Wait for reviewers to actually respond to the request
Symptoms: No webhook calls when request completesSolutions:
  • Verify callback_url is publicly accessible
  • Check webhook endpoint returns 200 status
  • Ensure webhook URL uses HTTPS
  • Test with webhook debugging tools like ngrok

Next Steps

🎉 Congratulations! You’ve successfully created your first loop and processed a request. Here’s what to explore next:

Production Checklist

Before deploying to production:
  • Test with multiple reviewers to ensure proper load distribution
  • Configure webhooks for automated response processing
  • Set up monitoring for request volume and response times
  • Train reviewers on your specific guidelines and criteria
  • Test timeout scenarios to validate default response handling
  • Implement retry logic for API calls and webhook handling
I