Hyper-usable HTTP client

In my opinion one design that absolutely nailed building services for the web was PHP so I built a client that takes inspiration from that. First lets start with a basic GET request.

from p2pd import *

# Every website today seems to want Javascript
# but its an example and demonstrates SSL too.
async def example():
    # Setup HTTP params.
    addr = ("www.google.com", 443)
    params = {"q": "lets search!"}
    path = "/search"
    conf = dict_child({
        "use_ssl": True
    }, NET_CONF)

    # Load interface and route to use.
    nic = await Interface()
    curl = WebCurl(addr, nic.route(IP4))

    # Make the web request.
    resp = await curl.vars(params).get(path, conf=conf)
    print(resp.req_buf)
    print(resp.out)

if __name__ == '__main__':
    async_test(example)

The above code shows how you can send $_GET params to a website. Loosely speaking: the vars method is where $_GET and $_POST go. There is a get, post, and delete method that cause these HTTP actions to occur. Here’s an example for POST:

from p2pd import *

async def example():
    nic = await Interface()
    addr = ("93.184.215.14", 80)
    curl = WebCurl(addr, nic.route(IP4))
    params = {"action": "upload"}
    payload = b"Data to POST!"
    resp = await curl.vars(
        url_params=params,
        body=payload
    ).post("/meow/cat.php")
    print(resp.out)

if __name__ == '__main__':
    async_test(example)

The design of this HTTP client results in copies of the objects properties being made for each call. This helps to encapsulate information and makes debugging a breeze. The return value of an API call is a new copy of the client with additional properties like a pipe, output, and more.

The HTTP client also supports custom HTTP headers. You can control underlying socket parameters with more detail by passing in a custom NET_CONFIG dictionary to the get, post, and delete methods. This structure is detailed at the bottom here: here.

class WebCurl():
    def __init__(self, addr, throttle=0, do_close=1, hdrs=[]):
        """
        throttle = a second timeout to delay a request by.
        do_close = whether or not the underlying pipe is closed (
            useful if you're streaming data and such.)
        hdrs = Custom HTTP request headers e.g.:
            [[b"user-agent", b"toxiproxy-cli"]]
        """
    #
    async def get(self, path, hdrs=[], conf=NET_CONF):
        pass
    #
    #...