One of the main benefits of using a content delivery network is reducing your infrastructure costs by greatly offloading your origin. However, customers often still want real-time CDN data regarding every hit to the CDN, as they would receive that CDN data if servicing all requests themselves. Highwinds‘ Request Receipt platform can help bridge that gap.
Request receipts send the equivalent of a normal log line back to a server that you control for each request that comes into the CDN, writing delivery receipt information directly to your servers. This can be especially useful for getting real-time information. You can utilize the data in a number of ways. Examples include writing it to a database, storing it in flat files, confirming full file delivery, using it for advanced business analytics, or modifying business behavior based on the information obtained.
Numerous pieces of information can be included in the delivery receipt, allowing a customer to construct a request receipt that is customized for their business needs.
How it works:
Request receipts are sent by making an HTTP(S) POST request to your receipt server endpoint URL, and receipt data resides in query string parameters with name/value pairs. You can deploy a receipt server on their side, or use a third-party log analysis product which often provides HTTP(S) endpoints for capturing log data.
The server only needs to be able to accept our HTTP/HTTPS request with query string or header data, so often a customer’s website can handle the task. For higher traffic volume, a resource dedicated to the task may be best. One example of a third-party analytics package that supports the Highwinds Request Receipt platform is Sumo Logic, which allows customers to feed the Highwinds receipt data directly into Sumo Logic for dashboards and advanced business analytics.
A typical request receipt is just a simple request to your web server endpoint, with the log data added as URL parameters to the request. Below a request receipt example is provided, with some of the most commonly used receipt parameters and definitions:
http://receipt.yourserver.com/receipt.php?event=%delivery.sendResult% httpstatus=%client.response.http.status% time=%client.request.startTime% ip=%client.ip% url=%client.request.fullUrl% Range=%client.request.headers.range.value% download=%delivery.totalBytesServed%/%delivery.attemptedBytesServed% file-size=%delivery.filesize% downloadtime=%delivery.contentTransferDuration% userAgent=%client.request.headers.user-agent.value% referer=%client.request.headers.referer.value% cachehit=%delivery.cacheHit% geocountry=%client.geoip.countryCode% origin=customer.origin.xxx.com X-HW=%client.response.headers.X-HW.value%
Data obtained from a typical request from the customer’s receipt server would be similar to the example below. Note: There was no range request or referrer specified for this particular request.
event=success httpstatus=200 time=1473352542 ip=7x.16x.19x.74 url=http://cds.r1e1a1f1.hwcdn.net/customerfile.html Range= download=974/974 file-size=618 downloadtime=1 userAgent=Pingdom.com_bot_version_1.4_(http://www.pingdom.com/) referer= cachehit=1&geocountry=US&origin=customer.origin.xxx.com&X-HW=1473352542.dop005.la2.t,1473352541.cds022.la2.c
Definitions of Commonly Used Fields for Request Receipts
%delivery.sendResult% : success | abort
%client.response.http.status% : HTTP Status code sent to the client. Ex: 200
%client.request.startTime% : Unix timestamp marking the beginning of the request
%client.ip% : IP in dot-notation of the client. Ex: 10.10.5.104
%client.request.headers.range.value% : Value of the range header in the client request
%delivery.totalBytesServed% : Total bytes downloaded to the client
%delivery.attemptedBytesServed% : Number of bytes that the CDN attempted to deliver
%delivery.contentTransferDuration% : Duration in milliseconds of the download
%origin.request.host% : Origin host name for requests if resulted in an origin pull
%delivery.filesize% : On-disk size of the requested asset
%client.request.fullUrl% : Full original URL of the client request. Ex:
%client.request.protocol% : Only the protocol of the client request. Ex: http/https
%client.request.host% : Only the host name of the client request. Ex: www.myvhost.com:80
%client.request.pathOnly% : Only the path of the client request. Ex: assets/i/image.jpg
%client.request.params% : Only the query string parameters of the client request. Ex: uniqueid=123456&username=bob
%client.request.method% : The HTTP method of the client request. Ex: GET
% client.request.headers.user-agent.value% : User-Agent http request header
%delivery.cacheHit% : Set to 0 if False or 1 if True
%client.geoip.countryCode% : Country code of end user
%client.response.headers.X-HW.value% : Value of Highwinds X-HW header
In addition to defined expansion variables, any client request headers or CDN response headers can also be added into the request receipt. For example, the Highwinds X-HW response header can be included in the receipt, providing valuable information on the PoP (point of presence) that delivered the content and whether the response was delivered from the cache. The parameters above are just a subset of values that can be included in the request receipt.
If real-time CDN data is not required for all requests, filters can be applied. For example, send a request receipt to your server only if the files delivered are video MP4s.
In addition to the request receipt platform outlined in this blog, Highwinds also provides detailed per-hit data in our raw logs, and the request receipt identifier can be added to these logs. Raw log data is archived for 45 days, so often customers choose to implement both: raw log for historical archival (45-day retention) and request receipt for real-time request information.
More information on raw logs is provided here:
If you’d like more information on the Highwinds CDN Data – Request Receipt platform, reach out to your Highwinds account manager, solutions engineer, or our support team. Once the service is enabled, a solutions engineer will work with you to get the request receipts implemented to fit your needs.
By Donna Hipp, Senior Solutions Engineer