Marmos
v juptune
v asn1
v lang
v ast
v Classes
v Enums
v ir
v parser
v Enums
v Structs
v core
v crypto
v keyexchange
v memory
v data
v alphabet
v buffer
v json
v builder
v parser
v Enums
v Structs
v serialise
v Structs
v Templates
v pem
v Enums
v Structs

Structs Http1ReaderBase

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
module juptune.http.v1;

struct Http1ReaderBase(
    SocketT
)
{
    /++ STRUCTS ++/
    
    ///
    struct MessageState
    {
        /++ VARIABLES ++/
        
        ///
        BodyEncoding bodyEncoding;
        ///
        size_t contentLength;
        ///
        size_t contentLengthRead;
        ///
        bool hasReadFirstChunk;
        ///
        Http1Version httpVersion;
        ///
        bool isBodyless;
        ///
        bool isRequest;
        ///
        Http1MessageSummary summary;
    }
    
    /++ ALIASES ++/
    
    ///
    alias Machine = ;
    ///
    alias StateMachine = Machine.;
    /// Checks if the headers have ended. 
    alias checkEndOfHeaders = ;
    /// ditto. 
    alias checkEndOfTrailers = ;
    /// Reads a single header. 
    alias readHeader = ;
    /// Ditto. 
    alias readTrailer = ;
    
    /++ FUNCTIONS ++/
    
    ///
    Result checkEndOfHeadersImpl(bool internal, bool trailers)(bool isEnd);
    ///
    Result fetchData(size_t bytesFetched, size_t savedCursor);
    /// Acknowledges that the message has been fully read, and returns the summary of the message. 
    Result finishMessage(Http1MessageSummary summary);
    ///
    Result processHeader(Http1Header header);
    /// Reads a single chunk of body data. 
    Result readBody(Http1BodyChunk bodyChunk);
    ///
    Result readBodyBytes(Http1BodyChunk chunk);
    ///
    Result readBodyChunked(Http1BodyChunk chunk);
    ///
    Result readBodyContentLength(Http1BodyChunk chunk);
    ///
    Result readChunkSize(size_t chunkSize, const char[] chunkExtension);
    ///
    Result readHeaderImpl(bool trailers)(Http1Header header);
    /// Reads the entire request line. This will configure the reader to be a request parser for the remainder of this message, until finishMessage is called. 
    Result readRequestLine(Http1RequestLine requestLine);
    /// Reads the entire response line. This will configure the reader to be a response parser for the remainder of this message, until finishMessage is called. 
    Result readResponseLine(Http1ResponseLine responseLine, Http1ReadResponseConfig responseConfig = Http1ReadResponseConfig.init);
    ///
    Result readUntil(ubyte delimiter)(ubyte[] slice);
    
    /++ ENUMS ++/
    
    ///
    enum BodyEncoding
    {
        ///
        FAILSAFE, 
        ///
        hasContentLength = 1 << 0, 
        ///
        isChunked = 1 << 1, 
        ///
        hasTransferEncoding = 1 << 2
    }
    ///
    enum State
    {
        ///
        FAILSAFE, 
        ///
        startLine, 
        ///
        headers, 
        ///
        maybeEndOfHeaders, 
        ///
        body, 
        ///
        trailers, 
        ///
        maybeEndOfTrailers, 
        ///
        finalise
    }
    
    /++ VARIABLES ++/
    
    ///
    ubyte[] _buffer;
    ///
    Http1Config _config;
    ///
    MessageState _message;
    ///
    size_t _pinCursor;
    ///
    uint _pinnedSliceIsAlive;
    ///
    size_t _readCursor;
    ///
    SocketT* _socket;
    ///
    StateMachine _state;
    ///
    size_t _writeCursor;
}

Documentation

A low-level reader for the HTTP/1.0 and HTTP/1.1 protocols, operating directly on a socket.

Performance

No explicit effort has been made to optimise this reader for performance, but it should be reasonably fast. No benchmarks have been done yet.

Correctness, safety, and readability have been the main priorities. This is not to say that optimisations will not be made in the future, but they are not a priority at the moment.

Memory wise the reader does not directly allocate heap memory as it uses user-provided buffers, however the kernal syscalls used for I/O can of course do whatever they want.

Buffer

The reader operates directly on a buffer provided by the user and does not directly allocate any memory.

For status lines, the buffer must be able to hold the entire status line.

For headers, the buffer must be able to hold the entire line for each singular header, however it does not need to be large enough to store the entire set of headers.

For bodies using chunked transfer-encoding, the buffer must be able to hold the entire chunk size and extension line. For the actual body data the buffer is used to determine the maximum amount of bytes that can be stored before the user is forced to process the data, to make way for the rest of the chunk.

For bodies using content-length, the buffer is used to determine the maximum amount of bytes that can be stored before the user is forced to process the data, to make way for the rest of the body.

Due to the general reuse of this buffer, returned data structures will often be "pinned", which means that the user code must process the data and then release the pin by destroying the returned data structure, prior to this reader fetching any more data from the socket.

Flow

This reader is a low-level, state-machine API, and thus requires quite a lot involvement from the user code, as well as a magical incantation of calls to properly parse requests and responses.

Any read function that returns a result will contain a valid HTTP response message that can be sent to the client as-is, if the result is an error. This is to allow the user code to quickly send an error response to the client without having to do any extra work.

This struct contains plenty of checks to ensure that the user code is not doing any illegal state transitions, and will also try its best to protect the user from themself in terms of memory safety.

RequestFlow

  1. Call readRequestLineto read the request line.

  2. Process the request line and destroy the returned Http1RequestLinestruct before continuing.

  3. While checkEndOfHeadersreturns false:

  4. Call readHeaderto read a header.

  5. Process the header and destroy the returned Http1Headerstruct before continuing.

  6. While readBody's return value's hasDataLeftis true:

  7. Process the body chunk and destroy the returned Http1BodyChunkstruct before continuing.

  8. Call finishMessageto finish the request, and if the Http1MessageSummary.connectionClosedis false, loop back to step 1 (if applicable), otherwise cease using this reader and socket for communication.

ResponseFlow

Responses are slightly different as in some cases it requires external context from a previous request/response as well as the fact responses can contain trailers for chunked bodies, thus requiring an extra processing stage.

  1. Call readResponseLineto read the response line.

  2. If the response is for a HEAD request, please ensure you set config.isBodylessto truefor correct handling.

  3. Process the response line and destroy the returned Http1ResponseLinestruct before continuing.

  4. While checkEndOfHeadersreturns false:

  5. Call readHeaderto read a header.

  6. Process the header and destroy the returned Http1Headerstruct before continuing.

  7. While readBody's return value's hasDataLeftis true:

  8. Process the body chunk and destroy the returned Http1BodyChunkstruct before continuing.

  9. While checkEndOfTrailersreturns false:

  10. Call readTrailerto read a trailer.

  11. Process the trailer and destroy the returned Http1Headerstruct before continuing.

  12. Call finishMessageto finish the response, and if the Http1MessageSummary.connectionClosedis false, loop back to step 1 (if applicable), otherwise cease using this reader and socket for communication.

Notes

Currently reading body data requires the use of the provided user buffer, but in the future a separate set of functions will be provided that will allow the user to provide a separate buffer specifically for body data.

Currently the reader does not provide a way to use gather/scatter I/O, but this is planned for the future, at least for body data.

This reader will never close the socket as it does not "own" the socket resource, it is up to the user to handle this. No side effects beyond reading data from the socket will occur.

You should ensure that messages are read in their entirety before responding to the client, as application errors are not the same as protocol errors, so the rest of the messages within the HTTP pipeline may be valid and should still be processed.

Security

The reader internally makes use of a type-system-based state machine to help ensure that bad state transitions can't occur.

See Http1Config.maxReadAttemptsfor a description of a potential DoS vector.

Issues

The reader is currently in a very early state, and is not yet ready for production use.

The current API is especially volatile and has no stability guarantees, as quite a lot of functionality is still missing or not yet exposed.

The reader currently does not support any form of compression.

The reader is not very extenstively tested.

Specific security concerns haven't been investigated or addressed yet.

The reader's currently priority is to handle the most common cases, and not to be a fully compliant for a while.

Specific differences between HTTP/1.0 and HTTP/1.1 are not fully implemented yet. HTTP/1.1 has been the main focus so far.

And of course there's all the other stuff I haven't thought of yet, such as more esoteric features and use cases.

Aliases

Name Description
Machine
StateMachine
checkEndOfHeaders

Checks if the headers have ended.

checkEndOfTrailers

ditto.

readHeader

Reads a single header.

readTrailer

Ditto.

Enums

Name Description
BodyEncoding
State

Functions

Name Description
checkEndOfHeadersImpl
fetchData
finishMessage

Acknowledges that the message has been fully read, and returns the summary of the message.

processHeader
readBody

Reads a single chunk of body data.

readBodyBytes
readBodyChunked
readBodyContentLength
readChunkSize
readHeaderImpl
readRequestLine

Reads the entire request line. This will configure the reader to be a request parser for the remainder of this message, until finishMessageis called.

readResponseLine

Reads the entire response line. This will configure the reader to be a response parser for the remainder of this message, until finishMessageis called.

readUntil

Structs

Name Description
MessageState

Variables

Name Description
_buffer
_config
_message
_pinCursor
_pinnedSliceIsAlive
_readCursor
_socket
_state
_writeCursor