Use Task.async_stream/3 and toy with the limit until you get a maximum that fits the hardware you’re on. Once you get that functioning your test will run (which is valuable in itself). Then if that number is unacceptable and runtime is too long, get a bigger box and see what it will give you. Selenium is slow and uses a lot of CPU as well. So its best to get something working as is first before springing to another box so you’ll have something to compare against when you do decide to increase the hardware (not to mention being able to justify or not justify the cost of doing so).

Also, consider starting with number of cores. Don’t try 100 or something crazy like that at first because it will never complete. Start with something with few resource constraints and inefficient and move up to efficient and max resources because once you go past that its just likely to waste your time.

Will you run this 500 js in the same browser? I assume you want to use the browser, as you’re refering to Drab.

Yes, exactly like in the async example - run the separate task for each process and collect results as it finishes.

This part might be a little bit tricky, if you want to do it in parallel