P5-paralleluseragent

Jul 20, 2023

Extension for LWP to allow parallel HTTP and FTP access

ParallelUserAgent or PUA for short is an extension of the existing libwww-5.x distribution. It allows you to connect to download several Web pages in parallel, without having to request each page one after the other.

Instead of retrieving each single page using LWPUserAgentrequest or simple_request methods, you first “register” all pages that you wish to download in parallel and then call PUA’s “wait” method, which will then make all the necessary connections and download the pages.

You can define callback routines which will be called whenever a connection is established, is cut off, new data comes in or a request finishes. The subroutines can be global for all requests you registered, or different for every single request.

Callbacks for example allow you to print status updates as the responses come in, or even immediately post follow-up requests based on the responses, all within the same single “wait” cycle.



Checkout these related ports:
  • Zope213 - Object-based web application platform Version 2.13
  • Zola - Fast static site generator
  • Zgrab2 - Fast Go application scanner
  • Zerowait-httpd - Lightweight and fast http server
  • Zenphoto - Simpler web photo gallery
  • Zend-framework - Framework for developing PHP web applications
  • Yuicompressor - The Yahoo! JavaScript and CSS Compressor
  • Ytdl - YouTube downloader written in Go
  • Yt-dlp - Command-line program for downloading videos from various platforms
  • Youtube_dl - Program for downloading videos from various services
  • Yourls - Your Own URL Shortener
  • You-get - Dumb downloader that scrapes the web
  • Yaws - Web server for dynamic content written in Erlang
  • Yarr - Yet another rss reader
  • Yarn - Package manager for node, alternative to npm (meta port)