-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathweb-server.html
125 lines (100 loc) · 5.54 KB
/
web-server.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>👨💻 C Web Server</title>
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/github-markdown-css/5.1.0/github-markdown.min.css">
<script src="https://cdn.jsdelivr.net/npm/marked/marked.min.js"></script>
<style>
body {
background-color: #212121;
margin: 0;
font-family: Arial, sans-serif;
}
.markdown-body {
background-color: #212121;
padding: 20px;
max-width: 800px;
margin: auto;
font-size: 14px;
color: white;
}
</style>
</head>
<body>
<div id="markdown-content" class="markdown-body"></div>
<script type="text/markdown" id="markdown-script">
# Building a simple HTTP Server in C
## Overview
A small web server in C to learn about multithreading, caching, thread-safe programming, etc.
Read about it on my <a href=https://github.com/zcroft27/web-server target="_blank">GitHub</a>.
**server.c Flow So Far**:
- Creates a new socket, binds it using the IPv4 option, and listens using the file descriptor for the new socket.
- Enters infinite while loop for accepting connections.
- Creates a pool of 10 worker POSIX threads, blocked by pthread_cond_signal, waiting to read requests.
- Accepts a connection and assigns a file descriptor for the new client.
- Enqueues a request and signals a thread to read the request from the client file descriptor into a buffer.
- Parses the buffer to extract the method (GET, POST, PUT, etc.) and the path/route.
- If the file is cached, serve from memory instead of reading using a system call.
- If read with a system call, cache current file data and metadata in least-recently-used (LRU) cache.
- Continue in loop, accepting up to 10 (arbitrary) connections concurrently.
## Interesting Features
- Initializes a pool of 10 worker threads before accepting connections, instead of creating
a new thread after accepting a connection.
- Uses a pthread_cond_signal to signal 1 worker thread at a time to serve a file.
- Locks shared-data access with a mutex to avoid datarace/deadlock issues.
- Utilizes a Least-recently-used (LRU) cache to store MAX_CACHE_QUEUE files in memory
to speed up serving files.
- Utilizes a dictionary alongside the LRU cache as a lookup table for cached nodes
for quicker access (Not actually O(1) access, TODO: implement hashmap).
- Enqueues requests from clients using a cute circular buffer I found that really confused me at first:
serve_file_args_t request_queue[MAX_QUEUE];
int queue_start = 0;
int queue_end = 0;
int queue_size = 0;
**On enqueue:**
request_queue[queue_end].clientfd = clientfd;
snprintf(request_queue[queue_end].filepath, sizeof(request_queue[queue_end].filepath), ".%s", filepath);
queue_end = (queue_end + 1) % MAX_QUEUE;
queue_size++;
**On dequeue:**
serve_file_args_t request = request_queue[queue_start];
queue_start = (queue_start + 1) % MAX_QUEUE;
queue_size--;
## What Did I Learn
I used pthread_cond_signal--instead of using a variable representing an threads in use--to notify
threads to start work on serving a file. pthread_cond_signal unblocks one thread (by default) that
is waiting on the pthread_cond_t passed into it, as opposed to pthread_cond_broadcast which unblocks
all threads waiting for the specified condition. (In hindsight I could have used a semaphore instead).
The dictionary/lookup table worked fine as as a singly linked-list, with a sentinel at the head for ease of removal/insertion. The queue of cache nodes (file path w/ data)
worked to implement as a doubly linked-list, with the queue having pointers to head and tail, both initially pointing to a 'dummy' node
for ease of removal/insertion. (I just learned this queue in systems and thought it would be cool).
Attempting to do this without using pthread_cond_signal and instead using a global variable to represent a number
of threads available led to weird data race issues, and pthread_cond_signal was a huge help.
Multithreading (with a capacity) seems like an effective way to handle concurrent requests,
although I am not sure if it would be better to use fork() to better isolate requests.
A simple way to test if your multithreading is working is to create a huge file, request it, and then
make another request concurrently.
I created long.txt:
shuf -i1-100000000 > long.txt
And requesting this file in ~10 tabs rapidly used 16gb of RAM before I killed the server.
## Intended Features/TODO
I got excited about caching and multithreading and forgot
about supporting other methods, serving more file types, etc.
- Modify serving based on method and file type.
- Log requests to server periodically.
- Refactor the LRU-cache to a dynamic pool of memory instead of allocating
2 byte chunks for each node in the cache queue.
- Experiment with using mmap instead of using a cache list on the heap.
- Instead of only caching the first 2 bytes of large files, do not cache them at all. (2 bytes is current arbitrary choice for max size).
- Modify dictionary for LRU cache to be a hashmap.
- Create client with etag-caching (caches and hashes content, sends hash to server, server compares hash to hash of real content, serves content if invalid)
- Switch to port 80.
</script>
<script>
const markdown = document.getElementById('markdown-script').textContent;
document.getElementById('markdown-content').innerHTML = marked.parse(markdown);
</script>
</body>
</html>