Your Own Personal WEB Server

Initialization

Upon startup server:

• changes working directory to ~/www/
• opens logfile (~/www/.log/access.log)
• loads config from ~/www/.conf/config
• creates listening socket (default TCP port is 8080)
• creates stage threads ('workers') and interstage channels (pipes)

Stage is described by struct worker in swebs.h header file:

	struct worker {

		char *name;
		void* (*action)(void*);

		char *in_queue_name;
		struct job_queue *in;

		char *out_queue_name;
		struct job_queue *out;

		char *aux_queue_name;
		struct job_queue *aux;
	};

Fields in this structure have following meanings:

• name - just a name of the stage, for ex. "Acceptor"
• action - a function that goes to pthread_create() as thread function
• in_queue_name - symbolic name of incoming queue
• in - queue of incoming jobs
• out_queue_name - symbolic name of outgoing queue
• out - outgoing jobs queue
• aux_queue_name -symbolic name of auxilary queue
• aux - auxilary queue, needed only by Sender

struct job_queue looks like this:

	struct job_queue {
		char *to_whom;
		int rdfd;
		int wrfd;
		int load;
	};

where

• to_whom - symbolic name of a pipe data consumer, for ex. 'To-Fetcher'
• rdfd - read end of a pipe
• wrfd - write end of a pipe
• load - number of jobs in the queue

The diagram below illustrates relations between stages:

				############
				# ACCEPTOR #
				############
				     |_________
                		     V	       |
				############   |
			  	# FETCHER  #   |
				############   |
				     |         |
				     V         |
				############   |
				#  SENDER  #___|
				############
				     |
				     V
				############
				#  LOGGER  #
				############

All worker types (stages) are described below.

Acceptor

Acceptor uses epoll_wait() for polling listening and data sockets and is responsible for:

• accepting clients
• reading HTTP requests from them
• allocating memory for newcoming job and initializing some fields in the structure, describing this job (remote IP address, socket fd and time at which request came
• parse this header and fill some fields in the job
• transfer a pointer to the job to Fetcher

struct job looks like this:

	struct job {
		struct connection *cnx;
		int sock;
		time_t when;
		int num;
		char who[IP_STR_LEN + 1];
		struct http_request_header hdr;

		int is_last;
		char content_type[64 + 1];
		char index_file[128 + 1];
		struct stat finfo;

		int response_hlen;
		int response_blen;
		size_t from_byte;
		size_t upto_byte;
		off_t offs;

		int fd;
		unsigned char *buf;
		size_t fetched;
		size_t sent;
		int buf_sent;
		int buf_total;

		int status;
		int done;
		nt pollinfd;
		char reason_500[16];
	}

Fetcher

There are following actions in the fetcher loop:

• reading the pointer to a job from "to-fetcher" pipe (note, a job may come either from acceptor or sender)
• load file chunk into memory and transfer job pointer to sender through "to-sender" pipe

Sender

Sender thread in its' loop does the following:

• calls epoll_wait() to poll sockets and "to-sender" pipe
• if that was message from fetcher, reads the pointer to a job and add a socket to the epoll set.
• if that was "ready for writing" event for a socket, send current chunk and increment chunk counter
• if it was not the last chunk of a file, transfer the job pointer back to fetcher, otherwise
• transfer the job pointer to logger

Logger

Logger is the final element in the stage chain. It

• reads job pointer from logger pipe
• updates log file
• frees memory used by job structure

Monitor

This one executes in the main thread of the server and in it's infinite loop:

• sleeps for a while
• inspects pipes load (how many jobs are there in each pipe)
• makes an attempt to clear one entry in file descriptor cache (see below)

File Descriptor Cache

In order not to waste file descriptors YOPS makes use of file descriptor cache (FDC). Caching file descriptors means that if 100 clients are downloading same file simultaneously, YOPS is using only one file descriptor for this file, not 100. For details see fd-cache.h in the source.