NOIV

Appearance

Troubleshooting ​

Common issues and solutions when using NOIV for API testing.

Installation Issues ​

Python Version Compatibility ​

Problem: NOIV installation fails with Python version errors.

bash
ERROR: Package 'noiv' requires a different Python: 3.7.0 not in '>=3.8'

Solution:

bash
# Check Python version
python --version

# Install Python 3.8+
# Ubuntu/Debian
sudo apt update
sudo apt install python3.9 python3.9-pip

# macOS with Homebrew
brew install python@3.9

# Use specific Python version
python3.9 -m pip install noiv

Package Installation Failures ​

Problem: pip installation fails with dependency conflicts.

bash
ERROR: Could not find a version that satisfies the requirement noiv

Solution:

bash
# Update pip and try again
python -m pip install --upgrade pip
pip install noiv

# Install from specific index
pip install --index-url https://pypi.org/simple/ noiv

# Use virtual environment
python -m venv noiv-env
source noiv-env/bin/activate  # Linux/macOS
# or
noiv-env\Scripts\activate  # Windows
pip install noiv

Permission Issues ​

Problem: Permission denied during installation.

bash
ERROR: Could not install packages due to an EnvironmentError: [Errno 13] Permission denied

Solution:

bash
# Install for current user only (Recommended)
pip install --user --upgrade noiv
export PATH="$HOME/.local/bin:$PATH"

# Or use virtual environment (recommended)
python -m venv venv
source venv/bin/activate
pip install noiv

Command-Line Issues ​

Command Not Found ​

Problem: noiv command not recognized after installation.

Solution:

bash
# Add ~/.local/bin to your PATH
export PATH="$HOME/.local/bin:$PATH"

# For permanent access, add to your shell config
# Example for zsh:
echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.zshrc
source ~/.zshrc

Configuration File Not Found ​

Problem: NOIV can't find test configuration files.

bash
Error: Test file 'tests.yaml' not found

Solution:

bash
# Use absolute path
noiv test /path/to/tests.yaml

# Check current directory
ls -la *.yaml *.json

# Create test file
touch api_tests.yaml

Test Configuration Issues ​

YAML Syntax Errors ​

Problem: YAML parsing errors in test files.

bash
Error: Invalid YAML syntax in line 15: mapping values are not allowed here

Solution:

yaml
# ❌ Incorrect - missing quotes
name: API Tests
base_url: https://api.example.com
body:
  message: Hello World!  # Missing quotes around value with space

# ✅ Correct
name: "API Tests"
base_url: "https://api.example.com"
body:
  message: "Hello World!"

YAML Validation Tips:

bash
# Validate YAML syntax
python -c "import yaml; yaml.safe_load(open('tests.yaml'))"

# Use proper indentation (2 spaces)
# Don't mix tabs and spaces
# Quote strings with special characters

Variable Resolution Errors ​

Problem: Variables not resolving in test execution.

bash
Error: Variable '{{api_key}}' not defined

Solution:

bash
# Set variable via command line
noiv test tests.yaml --variable api_key=your_key_here

# Set multiple variables
noiv test tests.yaml -v api_key=key123 -v user_id=user456

# Use environment variables
export API_KEY=your_key_here
noiv test tests.yaml --variable api_key=${API_KEY}
yaml
# Define variables in test file
variables:
  api_key: "default_key"
  base_url: "https://api.example.com"

# Use environment-specific variables
environments:
  development:
    variables:
      api_key: "dev_key"
  production:
    variables:
      api_key: "{{PROD_API_KEY}}"  # From environment

JSON Path Errors ​

Problem: JSONPath expressions not working.

bash
Error: JSONPath '$.user.profile.name' not found in response

Solution:

yaml
# ❌ Incorrect - response structure doesn't match
expect:
  body:
    user:
      profile:
        name: "John Doe"

# ✅ Check actual response first
# Response: {"data": {"user": {"name": "John Doe"}}}
expect:
  body:
    data:
      user:
        name: "John Doe"

# Use JSONPath assertions for flexible checking
assert:
  - jsonpath: "$.data.user.name"
    condition: "exists"
  - jsonpath: "$.data.user.name"
    condition: "equals"
    value: "John Doe"

JSONPath Testing:

bash
# Test JSONPath expressions online
# https://jsonpath.com

# Debug response structure
noiv test tests.yaml --verbose  # Shows full response

API Connection Issues ​

SSL Certificate Errors ​

Problem: SSL certificate verification failures.

bash
Error: SSL certificate verify failed: certificate verify failed: unable to get local issuer certificate

Solution:

bash
# Disable SSL verification (not recommended for production)
noiv test tests.yaml --no-verify-ssl

# Or in test configuration
yaml
tests:
  - name: Test with SSL disabled
    request:
      method: GET
      path: /endpoint
      verify_ssl: false
    expect:
      status: 200

Better solutions:

bash
# Update certificate bundle
pip install --upgrade certifi

# Set certificate bundle path
export SSL_CERT_FILE=/path/to/certificates.pem
export REQUESTS_CA_BUNDLE=/path/to/certificates.pem

Connection Timeout Issues ​

Problem: Requests timing out frequently.

bash
Error: Connection timeout after 30 seconds

Solution:

bash
# Increase global timeout
noiv test tests.yaml --timeout 60
yaml
# Set timeout per test
tests:
  - name: Slow endpoint test
    request:
      method: GET
      path: /slow-endpoint
      timeout: 120  # 2 minutes
    expect:
      status: 200

Network Connectivity Issues ​

Problem: Cannot reach API endpoints.

bash
Error: Failed to establish a new connection: [Errno -2] Name or service not known

Solution:

bash
# Test basic connectivity
ping api.example.com
nslookup api.example.com
curl -I https://api.example.com

# Check firewall/proxy settings
export HTTP_PROXY=http://proxy.company.com:8080
export HTTPS_PROXY=http://proxy.company.com:8080

Authentication Issues ​

Problem: Authentication failures in API requests.

bash
Error: 401 Unauthorized - Invalid authentication credentials

Solution:

bash
# Debug authentication
noiv test tests.yaml --verbose  # Shows full request headers

# Test authentication separately
noiv quick https://api.example.com/auth/login \
  --method POST \
  --body '{"username": "user", "password": "pass"}' \
  --verbose
yaml
# Common authentication patterns
tests:
  - name: Bearer Token Auth
    request:
      method: GET
      path: /protected
      headers:
        Authorization: "Bearer {{access_token}}"
    
  - name: API Key Auth
    request:
      method: GET
      path: /data
      headers:
        X-API-Key: "{{api_key}}"
    
  - name: Basic Auth
    request:
      method: GET
      path: /secure
      auth:
        type: basic
        username: "{{username}}"
        password: "{{password}}"

Test Execution Issues ​

Test Dependencies Not Working ​

Problem: Dependent tests not executing in order.

bash
Error: Test 'Get User' failed - depends on 'Create User' which failed

Solution:

yaml
# ✅ Correct dependency syntax
tests:
  - name: Create User
    request:
      method: POST
      path: /users
      body:
        name: "Test User"
    expect:
      status: 201
    extract:
      user_id: "$.id"
    
  - name: Get User
    depends_on: "Create User"  # Exact name match required
    request:
      method: GET
      path: /users/{{user_id}}
    expect:
      status: 200

Debugging dependencies:

bash
# Run with verbose output to see execution order
noiv test tests.yaml --verbose

# Run specific test only
noiv test tests.yaml --filter "Create User"

Variable Extraction Not Working ​

Problem: Variables not being extracted from responses.

bash
Error: Variable 'user_id' used but not extracted

Solution:

yaml
# ✅ Correct extraction syntax
tests:
  - name: Create User
    request:
      method: POST
      path: /users
      body:
        name: "Test User"
    expect:
      status: 201
    extract:
      user_id: "$.id"          # JSONPath to extract value
      auth_token: "$.token"    # Multiple extractions
      
  - name: Use Extracted Variable
    depends_on: "Create User"
    request:
      method: GET
      path: /users/{{user_id}}  # Use extracted variable
    expect:
      status: 200

Debug extraction:

bash
# Check response structure
noiv test tests.yaml --verbose  # Shows full response

# Test JSONPath extraction manually
echo '{"id": "123", "name": "John"}' | jq '.id'

Assertion Failures ​

Problem: Test assertions failing unexpectedly.

bash
Assertion failed: Expected status 200, got 404

Solution:

yaml
# Debug with verbose output
# noiv test tests.yaml --verbose

# Add multiple assertion types
assert:
  # Check if field exists first
  - jsonpath: "$.user.id"
    condition: "exists"
  
  # Then check value
  - jsonpath: "$.user.id"
    condition: "equals"
    value: "{{user_id}}"
  
  # Type checking
  - jsonpath: "$.user.age"
    condition: "type"
    value: "number"
  
  # Range checking
  - jsonpath: "$.user.age"
    condition: "between"
    min: 18
    max: 120

Performance Issues ​

Slow Test Execution ​

Problem: Tests running slower than expected.

bash
Test execution took 5 minutes for 10 tests

Solution:

bash
# Enable parallel execution
noiv test tests.yaml --parallel 4

# Reduce timeout for fast APIs
noiv test tests.yaml --timeout 10

# Remove unnecessary delays
noiv test tests.yaml --delay 0
yaml
# Optimize test configuration
tests:
  - name: Fast API Test
    request:
      method: GET
      path: /fast-endpoint
      timeout: 5  # Short timeout for fast endpoints
    expect:
      status: 200
      response_time_ms: 500  # Performance assertion

Memory Issues with Large Test Suites ​

Problem: High memory usage with large test suites.

bash
MemoryError: Unable to allocate array

Solution:

bash
# Run tests in smaller batches
noiv test tests.yaml --filter "auth.*"
noiv test tests.yaml --filter "user.*"

# Reduce parallel execution
noiv test tests.yaml --parallel 2

# Split large test files
# Instead of one large file, use multiple smaller files

Benchmark Performance Issues ​

Problem: Benchmark tests not reaching expected load.

bash
Warning: Only achieved 50 RPS instead of target 100 RPS

Solution:

bash
# Increase system resources
ulimit -n 65536  # Increase file descriptor limit

# Tune benchmark parameters
noiv benchmark tests.yaml \
  --users 100 \
  --ramp-up 120 \  # Longer ramp-up
  --rate-limit 50   # Lower rate limit

# Monitor system resources during tests
top
iostat 1

Import/Export Issues ​

Postman Import Failures ​

Problem: Postman collection import fails.

bash
Error: Invalid Postman collection format

Solution:

bash
# Ensure correct Postman export format
# In Postman: File → Export → Collection v2.1

# Validate JSON format
jq . postman_collection.json

# Use format specification
noiv import collection.json --format postman

# Check for unsupported features
# Pre-request scripts may need manual conversion

OpenAPI Import Issues ​

Problem: OpenAPI/Swagger import not generating expected tests.

bash
Warning: OpenAPI spec missing examples, using default values

Solution:

bash
# Include examples in generation
noiv import openapi.yaml --include-examples --generate-tests

# Validate OpenAPI spec first
# Use online validator: https://editor.swagger.io

# Check for required fields
# OpenAPI must have 'paths' and 'info' sections

Reporting Issues ​

HTML Report Generation Failures ​

Problem: HTML report generation fails.

bash
Error: Cannot generate report from empty results

Solution:

bash
# Ensure test results exist
ls -la *results*.json

# Check results format
jq . test_results.json | head -20

# Generate report with correct format
noiv test tests.yaml --format json --output results.json
noiv report results.json --output report.html

Report Display Issues ​

Problem: HTML report not displaying correctly in browser.

Solution:

bash
# Check browser console for JavaScript errors
# Try different browsers (Chrome, Firefox, Safari)

# Ensure report file is complete
wc -l report.html
tail -10 report.html  # Should end with </html>

# Regenerate with different template
noiv report results.json --template minimal --output simple_report.html

Environment-Specific Issues ​

Development vs Production Differences ​

Problem: Tests pass in development but fail in production.

bash
Tests: 15 passed in dev, 3 failed in prod

Solution:

yaml
# Use environment-specific configurations
environments:
  development:
    base_url: "http://localhost:3000"
    variables:
      timeout: 10
      retry_count: 1
      
  production:
    base_url: "https://api.example.com"
    variables:
      timeout: 30      # Higher timeout for production
      retry_count: 3   # More retries for production
bash
# Test with production-like settings in development
noiv test tests.yaml \
  --environment development \
  --variable timeout=30 \
  --retry 3

CI/CD Pipeline Issues ​

Problem: Tests pass locally but fail in CI/CD.

Solution:

yaml
# .github/workflows/tests.yml
- name: Install NOIV
  run: |
    python -m pip install --upgrade pip
    pip install noiv
    
- name: Wait for services
  run: |
    # Wait for API to be ready
    timeout 300 bash -c 'until curl -f http://localhost:8080/health; do sleep 5; done'
    
- name: Run tests with retries
  run: |
    noiv test tests.yaml \
      --environment ci \
      --retry 3 \
      --timeout 60 \
      --variable api_url=http://localhost:8080

Getting Help ​

Debug Mode ​

bash
# Enable maximum verbosity
noiv test tests.yaml --verbose

# Show request/response details
noiv quick https://api.example.com/endpoint --verbose

# Save debug output
noiv test tests.yaml --verbose > debug.log 2>&1

Community Support ​

  1. GitHub Issues: Report bugs or request features
  2. Documentation: Check the official documentation
  3. Examples: Browse example configurations

Common Error Patterns ​

Pattern: Connection refused

bash
# Check if service is running
curl -I https://api.example.com
netstat -an | grep :8080

Pattern: JSON decode errors

bash
# Validate API response format
curl -s https://api.example.com/endpoint | jq .

Pattern: Authentication token expired

bash
# Refresh tokens in test setup
setup:
  - name: Refresh Token
    request:
      method: POST
      path: /auth/refresh
      body:
        refresh_token: "{{refresh_token}}"
    extract:
      access_token: "$.access_token"

Pattern: Rate limiting

bash
# Add delays between requests
noiv test tests.yaml --delay 1000  # 1 second delay

# Or in test configuration
tests:
  - name: Rate Limited API
    request:
      method: GET
      path: /api/endpoint
    delay_ms: 1000  # Wait 1 second before this test

Remember to check the Quick Start guide for additional common questions and answers.

Released under the MIT License.

Copyright © 2025 NOIV Team