Skip to content

Parallel/Concurrent Programming

Thread Safety and Parallelism

Grapa is fully thread safe by design. All variable and data structure updates are internally synchronized at the C++ level, so you will never encounter crashes or corruption from concurrent access. However, if your program logic allows multiple threads to read and write the same variable or data structure, you may see logical race conditions (unexpected values, overwrites, etc.). This is a design consideration, not a stability issue. Minimize shared mutable state between threads unless intentional.

Only $thread() objects provide explicit locking and unlocking via lock(), unlock(), and trylock(). To protect access to a shared resource, create a $thread() lock object and use it to guard access. Calling .lock() or .unlock() on a regular variable (like an array or scalar) will return an error.

Canonical Example: 

lock_obj = $thread();
lock_obj.lock();
/* ... perform thread-safe operations on shared data ... */
lock_obj.unlock();

See Threading and Locking and Function Operators: static and const for details and best practices.

Grapa's $thread, $net, and map/reduce/filter features enable true parallelism, overcoming Python's GIL and enabling high-performance data processing.

Note: Grapa is fully thread safe, but if you share mutable state between threads, you are responsible for the logic. Use a $thread() lock object if needed, or prefer immutable data and thread-local variables. See Threading and Locking for examples and best practices.

Key Features for Parallel Programming:

  • True Parallelism: Bypass GIL limitations with native threads
  • Network Parallelism: Concurrent network operations
  • Functional Programming: Map/reduce/filter for data parallelism
  • Thread Safety: Built-in synchronization primitives

Example: Parallel Data Processing

/* Process large dataset in parallel */
process_chunk = op(chunk) {
    chunk.map(op(item) {
        /* Expensive computation */
        result = complex_calculation(item);
        {"input": item, "result": result};
    });
};

/* Split data and process in parallel */
parallel_process = op(data, num_threads) {
    chunk_size = data.len() / num_threads;
    chunks = [];

    i = 0;
    while (i < num_threads) {
        start = i * chunk_size;
        end = (i == num_threads - 1) ? data.len() : (i + 1) * chunk_size;
        chunks += data.mid(start, end - start);
        i += 1;
    };

    /* Process chunks in parallel */
    results = chunks.map(op(chunk) { process_chunk(chunk); });

    /* Combine results */
    results.flatten();
};

/* Example usage */
large_dataset = generate_test_data(1000000);
processed_data = parallel_process(large_dataset, 8);
("Processed " + processed_data.len().str() + " items in parallel").echo();

Example: Concurrent Network Operations with Automatic Completion

Grapa's .map() function provides sophisticated parallel execution with automatic completion and result collection:

/* Fetch multiple URLs concurrently with automatic completion */
fetch_urls = op(urls) {
    responses = urls.map(op(url) {
        try {
            response = $net().get(url);
            {"url": url, "success": true, "data": response.getfield("body")};
        } catch (error) {
            {"url": url, "success": false, "error": error.getfield("message")};
        };
    });
    responses;
};

/* Process API endpoints in parallel */
api_endpoints = [
    "https://api.example.com/users",
    "https://api.example.com/products", 
    "https://api.example.com/orders"
];

/* All 3 requests execute simultaneously */
/* Command completes when ALL requests finish */
/* Returns array of results in same order as input */
results = fetch_urls(api_endpoints);

successful = results.filter(op(r) { r.getfield("success"); });
failed = results.filter(op(r) { !r.getfield("success"); });

("Successful requests: " + successful.len().str()).echo();
("Failed requests: " + failed.len().str()).echo();

Key Benefits: - Parallel Execution: All requests start simultaneously - Automatic Completion: Command waits for ALL requests to finish - Result Collection: Returns array of results in same order as input - Performance: Total time = slowest request, not sum of all requests - Thread Safety: Each request runs independently with no race conditions

Example: Thread-Safe Data Processing

/* Thread-safe counter with locks */
thread_safe_counter = op() {
    counter = 0;
    lock = $thread();

    {
        "increment": op() {
            lock.lock();
            counter += 1;
            result = counter;
            lock.unlock();
            result;
        },
        "get": op() {
            lock.lock();
            result = counter;
            lock.unlock();
            result;
        }
    };
};

/* Use the thread-safe counter */
counter = thread_safe_counter();
threads = [];

/* Create multiple threads that increment the counter */
i = 0;
while (i < 10) {
    thread = $thread();
    thread.run(op() {
        j = 0;
        while (j < 100) {
            counter.increment();
            j += 1;
        };
    });
    threads += thread;
    i += 1;
};

/* Wait for all threads to complete */
threads.map(op(t) { t.join(); });
("Final counter value: " + counter.getfield().str()).echo();