Access Logs are now available from Clever Tools

Yes you read it right, you can now access your access logs. How you ask? With the latest release of Clever Tools. But before we see how it works, let me explain what access logs are.

Access logs contain all incoming requests to a web server. They are available in different formats. One of the most common being CLF which stands for Common Log Format. Here’s an example where you can find the IP of the client, date, HTTP verb, URL target, status code and size of the answer: - - [06/Feb/2020:07:59:22 +0100] "GET /aget/to/your/beautiful/website -" 200 1453

How to get your access logs

To get your access logs you need to use clever-tools and run this in a linked application:

clever accesslogs

You will get all the requests that came to this app in the last hour. This also work for most dedicated add-ons using --addon followed by the add-on id:

clever accesslogs --addon addon_41c5a60f-11c6-442a-9803-f4604fe8638b

You can also choose a date span using --after and --before like so:

clever accesslogs --after 2020-03-27 --before 2020-03-29

Any date you give must follow ISO 8601. You could for instance have a microsecond precision like this:

clever accesslogs --after 2020-03-29T11:01:02.120000Z --before 2020-03-29T11:01:02.123456Z

You can also change the format of the logs with -F (or --format):

clever accesslogs --after 2020-03-28 --before 2020-03-29 -F json

Here’s an example of JSON entry, it contains all of the information currently available for each request:

  "t": "2020-03-10T11:24:34.220324Z",
  "a": "app_87d748d4-3c7b-4baa-8f52-c748e1123ab3",
  "adc": "par-cleverapps-adc-n6",
  "o": "user_51f7098f-604e-4d9e-8fb6-4026fd1767bb",
  "i": "0805330e-cff6-4fd5-87c0-26974d8bcb4e",
  "ipS": "",
  "pS": 38566,
  "s": {
    "lt": 48.6719,
    "lg": 1.3002,
    "ct": "Aunay-sous-Crecy",
    "co": "FR"
  "ipD": "",
  "pD": 11810,
  "d": {
    "lt": 48.7967,
    "lg": 2.3338,
    "ct": "Cachan",
    "co": "FR"
  "vb": "GET",
  "path": "/status-codes/500",
  "bIn": 411,
  "bOut": 443,
  "h": "",
  "rTime": "3ms",
  "sTime": "115μs",
  "scheme": "HTTPS",
  "sC": 500,
  "sT": "Internal Server Error",
  "w": "WRK-01",
  "r": "b6bcab8a-c6d0-4546-9000-57b43d009a49",
  "tlsV": "TLS1.3"

You will find the details for every field in our access logs documentation.

What you can do with your logs

Now let’s say you want to know what are the most returned status codes from a log file, here’s how you can do it:

clever accesslogs --after 2020-03-10 --before 2020-03-20 -F json | jq .sC | sort | uniq -c | sort -nr

Here we use the JSON format and jq to select the field we want, then sort and uniq to get a proper output, which would yield something like:

  99999 200
     56 500
     33 404
      2 400
      1 403
      1 301
      1 202

The first column being the number of hits and the second being the status code.

Now for something a bit more practical, here’s a script that allows you to split access logs for each day between two dates:



startdate=$(date -I -d "$input_start") || exit -1
enddate=$(date -I -d "$input_end")     || exit -1

while [ "$d" != "$enddate" ]; do 
  clever accesslogs --alias prod --after $d --before $(date -I -d "$bv d + 1 day") -F clf > access-$d.out
  d=$(date -I -d "$d + 1 day")

It’s hard to visualize so much data if you don’t have the proper tools. So I really invite you to try GoAccess. Combine the script above and this command goaccess access* -o report.html --log-format=COMMON --anonymize-ip. You will get this neat, anonymized HTML report:

Demonstrating the filter
Viusalize CLF logs with GoAccess

There are of course lots of other tools you can use to visualize logs, like our latest Elastic add-on for instance 🙂 Or you can go straight into the metrics tab of your application or add-on and start playing with Warp10. This is actually what we use when you see the live map under the Overview tab of your application.

We hope you’ll find this new feature practical and as always welcome your input to make this better. Happy coding!


À lire également

Deploy llama, mistral, openchat or your own model on Clever Cloud

If AI has been in the news recently, it's mainly through large language models (LLMs), which require substantial computing power for training or inference. Thus, it's common to see developers using them via turnkey APIs, as we did in a previous article. But this is changing thanks to more open and efficient models and tools, which you can easily deploy on Clever Cloud.

Our consortium, InfrateX, wins Simpl

The InfrateX Consortium, led by Sopra Steria with NTT Data and including Clever Cloud as a member, has won the significant contract Simpl.
Company Press

Our end-of-year events

The end of the year is already shaping up to be full of events, trade…