Fail2ban is a tool which you can use to reduce the impact of attacks on your servers. Typically you configure it to monitor a log file for suspicious activity.  Then once the activity crosses a threshold you can have it take an action, such as block the source IP address in your firewall.  It’s a good way to stop attacks early but doesn’t entirely prevent them.

Why use it to protect WordPress

Due to it’s popularity, WordPress is often the target of automated attacks.  We often see bruteforce attacks targeting xmlrpc.php or wp-login.php, these rely on making a huge number of requests in the hope that one will eventually be successful.  Using strong passwords, especially for accounts with admin access is important to reduce the risk from attacks.  Fail2ban can be used to slow attackers down.  This helps for two reasons: it makes them less likely to succeed; it reduces the load on the server caused by processing these requests.

Other options

• Using an external Web Application Firewall (WAF) such as Sucuri or Cloudflare.
• Using webserver configuration options to limit requests such as limit_req in Nginx.
• WordPress security plugins such as Wordfence.

Blocking an attack as far upstream as possible is always advantageous to save resources so we would typically favour a Web Application Firewall or Webserver configuration over using fail2ban or a WordPress plugin however every site, server and customer is different so it will depend on the exact configuration used and required.

Install & config fail2ban (for Ubuntu)

sudo apt-get install fail2ban

Fail2ban works by having a jail file which references the log file, a filter and an action.  By default fail2ban will protect sshd.  If you are restricting SSH access in another way then you might want to turn this off.  You can do so by creating the following as /etc/fail2ban/jail.local

[sshd]
enabled = false

Fail2ban doesn’t come with a filter for WordPress.  I’d like to credit these two articles for providing a good starting point.  We see requests to ‘//xmlrpc.php’ (note the double slashes) fairly frequently so tweaked the below to also flag them.  As this is detecting any requests to wp-login/xmlrpc it will flag legitimate admin users when they login etc.  We’ll look to account for this with the jail configuration.

You can create the filter as /etc/fail2ban/filter.d/wordpress.conf

[Definition]
failregex = ^<HOST> .* "(GET|POST) /+wp-login.php
            ^<HOST> .* "(GET|POST) /+xmlrpc.php

The jail file has most of the configuration options:

• logpath, in this case the path to your Apache access log
• action, adjust this if you’re using a different firewall or want to be sent email instead, you can see available options in /etc/fail2ban/action.d/
• maxretry, the number of requests within findtime seconds to ban after
• bantime, the number of seconds to ban for, with this action how long they’re blocked in iptables

As mentioned, the filter will catch both malicious and legitimate users.  We’re configuring maxrety fairly high and bantime relatively low to minimise the probability and impact if we do block a legitimate user.  Whilst this allows attackers to make roughly one request every ten seconds this is a fraction of what they’d make without fail2ban.

You can create the jail file as /etc/fail2ban/jail.d/wordpress.conf

[wordpress]
enabled = true
port = http,https
filter = wordpress
action = iptables-multiport[name=wordpress, port="http,https", protocol=tcp]
logpath = /var/log/apache2/all-sites-access.log
maxretry = 12
findtime = 120
bantime = 120

Restart fail2ban to load in your new config

sudo systemctl restart fail2ban

Checking your fail2ban set-up

Show the jails

sudo fail2ban-client status

Show info about the WordPress jail

sudo fail2ban-client status wordpress

List the f2b-wordpress iptables chain (this will be created the first time fail2ban blocks an IP and contain blocked IPs)

sudo iptables -v -n -L f2b-wordpress
sudo ip6tables -v -n -L f2b-wordpress

View the log to see any IPs banned or flagged

sudo less /var/log/fail2ban.log

Feature image by Jussie D.Brito licensed CC BY-SA 2.0.

The post How to set-up fail2ban for a WordPress site appeared first on Dogsbody Technology.

Optimisations are always important, especially at scale; we’re always looking for the cleanest way to do things and redirects are definitely one of the easiest and quickest wins.

Redirects are used when you want to send users to a different URL based on the requested page.  Some popular reasons to do this are:

• Redirect users from a short memorable URI to a corresponding page
• Keep old URLs working after migrating from one bit of software to another
• Redirect discontinued product pages (returning 404s) to the most closely related page

Redirects can typically be done in using rewrite or  return with the addition of map for large arrays of redirects. Lets look at the differences….

Using ‘rewrite’ for redirects

Rewrite isn’t all bad, it’s very powerful but with that comes a large CPU overhead as it uses regex by default.

Consider the following rule to redirect browsers accessing an image to a separate subdomain; perhaps a CDN…

rewrite ^/images/(*.jpg)$ http://static.example.com/$1 permanent;

Using ‘return’ for redirects

A redirect is as simple as returning a HTTP status code so why not do just that with the return option.

If you have to redirect an entire site to a new URL then it can be simply done with…

server {
    server_name oldsite.example.com;
    return 301 $scheme://newsite.example.com$request_uri;
}
server {
    server_name newsite.example.com;
    # [...]
}

… remember that http and https configs can easily be separated to allow the http site to only forward on requests ensuring that it’s impossible for anyone to access your site insecurely…

server {
    listen 80;
    server_name www.example.com;
    return 301 https://www.example.com$request_uri;
}
server {
    listen 443;
    server_name www.example.com;
    # [...]
}

Using ‘map’ for redirects

If you’ve only got a handful of redirects then using rewrite to do this isn’t a big deal.  Once you’re talking about hundreds or thousands of redirects it’s worth looking to use a map.

Here’s an example configuration using a map and return directive to set-up two redirects.

Depending on how many URLs you’re redirecting you’ll need a different value for map_hash_bucket_size.  I’d stick with powers of 2, a config test (nginx -t) will warn you if your value is too small.
$uri is an NGINX variable.
$new_uri is the new variable that we’re creating, if you’ve got multiple sites you’ll need to use different variable names.

map_hash_bucket_size 128;
map $uri $new_uri {
    /test-to-rewrite /test-rewritten;
    /test2-to-rewrite /test2-rewritten;
}

server {
    server_name example.com;
    if ($new_uri) {
        return 301 $new_uri;
    }
#...
}

For reference, here’s the equivalent syntax using rewrite.

server {
    server_name example.com;
    rewrite ^/test-to-rewrite$ /test-rewritten permanent;
    rewrite ^/test2-to-rewrite$ /test2-rewritten permanent;
#...
}

Testing map performance

Before testing this my expectation was that there would be a cross over point after which using a map was beneficial.  What I found was that a map is suitable for any number of redirects but it’s not until you have a large number that it’s essential.

Test conditions

• Ubuntu 18.04 on a AWS c5.large
• nginx v1.18.0
• Used ApacheBench to make 10k local requests with 500 concurrent connections

I ran each test 3 times and used the average in the numbers below.  I was measuring:

• Time taken to complete all of the requests
• Memory usage with NGINX stopped, running and running after the test

I’m happy this can show the relationship between the number of redirects and the performance differences between a map and rewrite.  Though it won’t represent your exact set-up.

Single redirect

The first thing I tested using both the rewrite and map configuration was a single redirect.

The time to complete these requests was comparable.  Both round to 0.42s with the map being a few hundredths of a second slower.  I believe this is within the errors of my testing.  The memory usage of running NGINX was too close to separate, fluctuating between 3-6M.

Large (83k) redirects

After that I tested what I’d consider the other extreme.  I pulled 83k lines out of the dictionary file and set-up redirects for them.  I ran the test against multiple different words.  This shows how the number of preceding redirects affects the time to complete the requests.

• Abdomen – which appears early in my redirects had comparable times (the average over the 3 test runs is actually identical)
• Absentee – the 250th redirect took 0.66s with rewrite rules and 0.42s with the map
• Animated – the 2500th redirect took 4.43s with rewrite rules and 0.42s with the map
• Evangelising – the 25000th redirect took 27.86s with rewrite rules and 0.42s with the map
• Zing – which appears late in my redirects took 57.54s with rewrite rules and 0.42s with the map
• The 404 page I was testing – this represents load times of any small static page below the redirects in the config, it took 56.89s with rewrite rules and 0.43s with the map

The time to process each different redirect was effectively constant when using a map.  Using rewrite the time to process a redirect is proportional to the number of rewrite rules before it.

Memory usage was also noticeably different.  I found that the increase due to running NGINX was ~16M for the map and ~66M for the rewrites.  After the test had run this increased by a few megabytes to ~19M and ~68M.

250 redirects

I wanted to check if the sheer number of rewrites was slowing things down.  I cut the config down to just the first 250 redirects.  This significantly reduced memory usage.  The time taken for requests to the Absentee redirect was negligibly different from when there were 83k redirects.

Fewer requests

I ran an extra test with 100 requests (rather than 10k) and 5 concurrent connections (rather than 500).  This is also a closer approximation to a single user accessing a webpage.  The time taken to access the Zing redirect was 0.55s (rather than 57.65s).  I’m happy that this shows the time for a single request is effectively constant.

Conclusion

For a large number of redirects a map is both faster and more memory efficient.  For a small number of redirects either rewrite or using a map is acceptable.  Since there’s no discernible disadvantage to a map and you may need to add more redirects in the future I’d use a map when possible.

Give us a shout if we can help you with your NGINX setup.

Feature image by Special Collections Toronto Public Library licensed CC BY-SA 2.0.

The post NGINX – Optimising Redirects appeared first on Dogsbody Technology.

In this article, I want to show you a couple of the first places you should head, when you need to pinpoint bottlenecks or tweak the MySQL configuration.

MySQL slow log

The slow log, will log any queries that take longer than a given number of seconds.

This can help you to identify poorly written or demanding queries.  You can then refactor them or use concepts like “indexes” to speed them up.

It’s often helpful to start with a high “long query time”, to just flag up the longest queries and then gradually reduce it, as you deal with each one in turn.

To enable the slow log, create the following /etc/mysql/mysql.conf.d/mycustomconfigs.cnf or add the following lines to your my.conf file…

[mysqld]
slow_query_log=true
long_query_time=1
slow-query-log-file=/var/log/mysql/mysql-slow

… then restart MySQL, to load in the new values.

Improve the query

Once you’ve found a slow query, it’s worth considering if there is a simpler way to get the same information.

If you can improve the performance of the query you might be able to skip looking into why the old one was slow.

Explain/Describe

If you’re still looking to improve your query, we need to dig into how MySQL is actually running the query and why it’s slow.  This will give us a better idea how to fix it.

For these examples I ran a few basic queries against this auto generated employee data.

Let’s suppose your slow query is:

SELECT AVG(hire_date) FROM employees WHERE emp_no IN (SELECT emp_no FROM dept_manager)

To see how it will be executed, prefix it with EXPLAIN:

mysql> EXPLAIN SELECT AVG(hire_date) FROM employees WHERE emp_no IN (SELECT emp_no FROM dept_manager);
+----+-------------+--------------+------------+--------+---------------+---------+---------+-------------------------------+------+----------+------------------------+
| id | select_type | table        | partitions | type   | possible_keys | key     | key_len | ref                           | rows | filtered | Extra                  |
+----+-------------+--------------+------------+--------+---------------+---------+---------+-------------------------------+------+----------+------------------------+
| 1  | SIMPLE      | dept_manager | NULL       | index  | PRIMARY       | PRIMARY | 8       | NULL                          | 24   | 100.00   | Using index; LooseScan |
| 1  | SIMPLE      | employees    | NULL       | eq_ref | PRIMARY       | PRIMARY | 4       | employees.dept_manager.emp_no | 1    | 100.00   | NULL                   |
+----+-------------+--------------+------------+--------+---------------+---------+---------+-------------------------------+------+----------+------------------------+
2 rows in set, 1 warning (0.00 sec)

It’s worth having a quick look at the documentation on the output format.  Three of the columns to check are:

• key – is the index that will be used, the one you’d expect?
• rows – is the number of rows to be examined as low as possible?
• filtered – is the percentage of results being filtered as high as possible?

Suppose we’re regularly making the following query:

SELECT * FROM employees WHERE gender = 'M'

This has type: ALL meaning that all rows in the table will be scanned.

It therefore makes sense here to add an index.

After doing so, the type changes to ref – MySQL can simply return the rows matching the index rather than checking every row.

As you’d expect this halves the number of rows:

mysql> EXPLAIN SELECT * FROM employees WHERE gender = 'M';
+----+-------------+-----------+------------+------+---------------+------+---------+------+--------+----------+-------------+
| id | select_type | table     | partitions | type | possible_keys | key  | key_len | ref  | rows  | filtered  | Extra       |
+----+-------------+-----------+------------+------+---------------+------+---------+------+--------+----------+-------------+
| 1  | SIMPLE      | employees | NULL       | ALL  | NULL          | NULL | NULL    | NULL | 299025 | 50.00    | Using where |
+----+-------------+-----------+------------+------+---------------+------+---------+------+--------+----------+-------------+
1 row in set, 1 warning (0.00 sec)

mysql> CREATE INDEX gender ON employees(gender);
Query OK, 0 rows affected (0.97 sec)
Records: 0 Duplicates: 0 Warnings: 0

mysql> EXPLAIN SELECT * FROM employees WHERE gender = 'M';
+----+-------------+-----------+------------+------+---------------+--------+---------+-------+--------+----------+-------+
| id | select_type | table     | partitions | type | possible_keys | key    | key_len | ref   | rows   | filtered | Extra |
+----+-------------+-----------+------------+------+---------------+--------+---------+-------+--------+----------+-------+
| 1  | SIMPLE      | employees | NULL       | ref  | gender        | gender | 1       | const | 149512 | 100.00   | NULL  |
+----+-------------+-----------+------------+------+---------------+--------+---------+-------+--------+----------+-------+
1 row in set, 1 warning (0.01 sec)

Third Party Tools

I should mention, that there are a bunch of tools that can help you find bottlenecks and really hone your MySQL database optimization techniques.

For websites written in PHP, we’re big fans of New Relic APM. This tool will allow you to sort pages based on their load time.  You can then dig deeper into whether the application code or database queries have the most room for improvement.

Once you’ve narrowed things down, you can start implementing improvements.

It’s worth having a search for other application monitoring providers to see if tools such as Datadog or DynaTrace better suit you.

MySQL Tuner

MySQL Tuner is a tool which looks at a database’s usage patterns to suggest configuration improvements.  Make sure you run it on a live system, that has been running for at least 24 hours.  Otherwise it won’t have access to enough data to make relevant recommendations.

Before you download and use the tool I’ll echo it’s warning:

Once you run MySQL Tuner, it will login to the database, read a bunch of metrics and print out some recommendations.

It’s worth grouping the related ones and reading up on any you haven’t come across.  After that you can test out changes in your staging environment.

One common improvement is to set skip-name-resolve.  This saves a little bit of time on each connection by not performing DNS lookups.  Before you do this make sure you aren’t using DNS in any of your grant statements (you’re just using IP addresses or localhost).

Your friendly SysAdmins

Of course, we are also here to help and regularly advise customers on changes that can be made to their infrastructure.

Give us a shout if you think we can help you too.

Feature image by Joris Leermakers licensed CC BY-SA 2.0.

The post A short guide to MySQL database optimization appeared first on Dogsbody Technology.

For security reasons, it is best practice to disable TLS 1.0 and TLS 1.1, but before you do this you may need to weigh up the risks to traffic from old browsers.

After disabling TLS 1.0 and TLS 1.1 any visitors using old browsers won’t be able to access your site.  If you are accepting credit card payments through your website then your customers security is more important but if you have a public information site then this may not be the case.

Don’t I always want the best security?

Please don’t get us wrong. We are NOT advocating blindly reducing security. This post is very much a response to customers that come to us wanting changes that will break their sites in order to get a perfect score or tick a compliance box. We can usually come up with a best of both worlds once we show the exact implications of the change.

What’s the fuss about?

GlobalSign’s What’s Behind the Change? paragraph sums this up nicely:

Various vulnerabilities over the past few years (e.g., BEAST, POODLE, DROWN…we love a good acronym, don’t we?) have had industry experts recommending disabling all versions of SSL and TLS 1.0 for a while now. PCI Compliance was another driving factor. On June 30, 2018, the PCI Data Security Standard (DSS) required that all websites needed to be on TLS 1.1 or higher in order to comply.

The RFC 7525 from 2015 stipulates that implementations should not use TLS 1.0 or TLS 1.1:

   o  Implementations SHOULD NOT negotiate TLS version 1.0 [RFC2246];
      the only exception is when no higher version is available in the
      negotiation.
      Rationale: TLS 1.0 (published in 1999) does not support many
      modern, strong cipher suites.  In addition, TLS 1.0 lacks a per-
      record Initialization Vector (IV) for CBC-based cipher suites and
      does not warn against common padding errors.
   o  Implementations SHOULD NOT negotiate TLS version 1.1 [RFC4346];
      the only exception is when no higher version is available in the
      negotiation.
      Rationale: TLS 1.1 (published in 2006) is a security improvement
      over TLS 1.0 but still does not support certain stronger cipher
      suites.

Qualys SSL Labs have reduced their grading for servers which support TLS 1.0 or TLS 1.1

Assessing the risk

Who won’t be able to access my website if I disable TLS 1.0 or TLS 1.1? Generally speaking browsers before 2013 will have trouble.  Most popular clients affected are old Android phones and old versions of Windows with Internet Explorer 10.  For the exact Android versions and other affected clients this is a nice breakdown.  As you’d expect the number of visitors with these old clients will vary according to your user base.  It’s best you check your site’s analytics to inform your decision.

Again, you can take into account how important encryption is for your website.  For example, at the time of writing it’s interesting to note that paypal.com has removed support for TLS 1.0 & 1.1 whilst google.com has not.

Summary

So what does this mean?  Lets give some examples…

If security is important to you; perhaps you have an e-commerce site taking payments or you are a IT consultancy like ourselves where people wish to share private information. You must disable old SSL/TLS protocols so that the only way people can communicate with your site is as secure as possible.

If accessibility is important to you; perhaps you are trying to share public information, be it a marketing or public resources site. It maybe worth supporting old protocols to allow your message to be shared as wide as possible.

Remember; it maybe typically called a sales “funnel” but traffic doesn’t have to end up in just one place. Users not supporting the right levels of security can be redirected to alternative pages where they can be contacted in other ways.  Why lose a sale when you don’t have to!

We’ve intentionally painted with broad strokes in this blog post.  We’re happy to give specific advice if you contact us and feel free to leave a comment

The post Removing support for TLS 1.0 and TLS 1.1 appeared first on Dogsbody Technology.

Computers need a way to represent characters in binary form – as a series of ones and zeros – equivalent to the dots and dashes used by Morse code.

ASCII

A widely used way for computers to encode information, is ASCII (American Standard Code for Information Interchange), created in the 1960’s.

ASCII defines a string of seven ones and zeros that represent the letters A-Z, upper and lowercase as well as numbers 0-9 and common symbols. 128 characters in total.

8 bit encoding

As you’d expect, ASCII is well suited for use in America, however it’s missing many characters that are frequently used in other countries.

For example, it doesn’t include characters like é or £ & €.

Due to ASCII’s popularity, it’s been used as a base to create many different encodings.  All these different encodings add an extra eighth bit, doubling the possible number of characters and using the additional space for characters used by differing groups …

• Latin 1 – Adds Western Europe and Americas (Afrikaans, Danish, Dutch, Finnish, French, German, Icelandic, Irish, Italian, Norwegian, Spanish and Swedish) characters.
• Latin 2 – Adds Latin-written Slavic and Central European (Czech, Hungarian, Polish, Romanian, Croatian, Slovak, Slovene) characters.
• Latin 3 – Adds Esperanto, Galician, Maltese, and Turkish characters.
• Latin 4 – Adds Scandinavia/Baltic, Estonian, Latvian, and Lithuanian characters (is an incomplete predecessor of Latin 6).
• Cyrillic – Adds Bulgarian, Byelorussian, Macedonian, Russian, Serbian and Ukrainian characters.
• Arabic – Adds Non-accented Arabic characters.
• Modern Greek – Adds Greek characters.
• Hebrew – Adds Non-accented Hebrew characters.
• Latin 5 – Same as Latin 1 except for Turkish instead of Icelandic characters
• Latin 6 – Adds Lappish/Nordic/Eskimo languages. Adds the last Inuit (Greenlandic) and Sami (Lappish) letters that were missing in Latin 4 to cover the entire Nordic area characters.
• etc.

All of this still doesn’t give global coverage though! There’s also an issue due to the inability of using different encodings on a single document, should you ever need to use characters from different character sets.

We need an alternative …

Unicode

Unicode seeks to unify all the characters into one set.

This simplifies communication, as everyone can use a shared character set and doesn’t need to convert between them.

Unicode allows for over a million characters!

One of the most popular ways to encode Unicode, is as UTF-8.  UTF-8 has a variable width. Depending on the character used to encode, either 8, 16, 24 or 32 bits are used.

For characters in the ASCII character set, only 8 bits need to be used.

Another way to encode Unicode is UTF-32, which always uses 32 bit. This fixed width is simpler, but causes it to often use significantly more space than UTF-8.

Emoji

You probably don’t need telling, but Emoji are picture characters.

For a long time, knowledge workers have created smiley faces and more complex emoticons using symbols.

To take this a step further, emoji provide a wealth of characters.

The data transferred is always the same, but the pictures used differ between different platforms. Depending on the device, you’re viewing this, our smiley face emoji, ?, will look different.

The popularity of emoji has actually helped push Unicode support, which includes emoji as part of its character set.

I’ve pulled out a few recently added ones and you can see more on the Unicode website.

U+1F996 added in 2017 – T-Rex ?
U+1F99C added in 2018 – Parrot ?
U+1F9A5 added in 2019 – Sloth ?

Feature image by Thomas licensed CC BY-SA 2.0.

The post Exploring character encoding types appeared first on Dogsbody Technology.

This means any Python code that’s still on 2.x needs updating to Python 3.  Any code that isn’t moved over won’t receive security updates so will inevitably become insecure.

Identify your code

If you’ve got a lot of code it’s worth taking the time to check what’s where and which version of Python it’s using.

Python 3 was released at the end of 2008.  Adoption has been slow, a factor has been that all of your dependencies need to support Python 3 before you can.  Now that we’re over 10 years down the road this is much less likely to be an issue.

You can start off by checking code that has been written more recently.  Hopefully this will have been written for Python 3.  A survey by JetBrains shows that between 2017 and 2018 the number of developers that mostly used Python 2 fell from 25% to just 16%.  It’s also interesting to note the divide between use cases.  Data science having better adoption than both web and dev-ops.

Don’t forget old code

Unfortunately the numbers above are for code that developers are writing now.  We’re also concerned with code that was written many years ago and hasn’t recently had any major changes.  Looking at the number of packages downloaded instead of what developers are mostly using gives a different picture.  The numbers are closer to 50/50 with the trend between data science and dev-ops still clear.  TensorFlow is most often downloaded for Python 3 whilst botocore is heavily Python 2.  Boto is heavily used in API access to cloud providers such as AWS.

If all of your recent code is Python 3 it’s worth having a good dig around for places old code might be hiding.

What are the steps to update to Python 3?

• The first step to update code is to make sure any packages you’re using support Python 3.  A tool such as caniusepython3 should show you where the issues are.
• After that depending on the complexity of your code you can update it by hand or use a tool such as Futurize to help with the conversion .

A key part of smoothly updating is to have a good testing process so you can quickly find and fix the bits that unexpectedly break.  See the porting guide for more info.

Feature image by See1,Do1,Teach1 licensed CC BY 2.0.

The post Python 2 will go end of life on 01 Jan 2020 appeared first on Dogsbody Technology.

What is Tripwire

Open Source Tripwire is a popular host based intrusion detection system (IDS).  It is used to monitor filesystems and alert when changes occur.  This allows you to detect intrusions or unexpected changes and respond accordingly.  Tripwire has great flexibility over which files and directories you choose to monitor which you specify in a policy file.

How does it work

Tripwire keeps a database of file and directory meta data.  Tripwire can then be ran regularly to report on any changes.

If you install Tripwire from Ubuntu’s repo as per the instructions below a daily cron will be set-up to send you an email report.  The general view with alerting is that no news is good news.  Due to the nature of Tripwire it’s useful to receive the daily email, that way you’ll notice if Tripwire gets disabled.

Before we start

Before setting up Tripwire please check the following:

• You’ve configured email on your server.  If not you’ll need to do that first, we’ve got a guide.
• You’re manually patching your server.  Make sure you don’t have unattended upgrades running (see the manual updates section) as unless you’re co-ordinating Tripwire with your patching process it will be hard for you to distinguish between expected and unexpected changes.
• You’re prepared to put some extra time into maintaining this system for the benefit of knowing when your files change.

Installation on Ubuntu

sudo apt-get update
sudo apt-get install tripwire

You’ll be prompted to create your site and local keys, make sure you record them in your password manager.

In your preferred editor open /etc/tripwire/twpol.txt

The changes you make here are based on what you’re looking to monitor, the default config has good coverage of system files but is unlikely to be monitoring your website files if that’s something you wanted to do.  For example, I’ve needed to remove /proc and some of the files in /root that haven’t existed on systems I’ve been monitoring.

Then create the signed policy file and then the database:

sudo twadmin --create-polfile /etc/tripwire/twpol.txt
sudo tripwire --init

At this point it’s worth running a check. You’ll want to make sure it has no errors.

sudo tripwire --check

Finally I’d manually run the daily cron to check the email comes through to you.

sudo /etc/cron.daily/tripwire

Day to day usage

Changing files

After you make changes to your system you’ll need to run a report to check what tripwire sees have changed.

sudo tripwire --check

You can then update the signed database.  This will open up the report allowing you to check you’re happy with the changes before exiting.

sudo tripwire --update -r /var/lib/tripwire/report/$HOSTNAME-yyyyMMdd-HHmmss.twr

You’ll need your local key in order to update the database.

Changing policy

If you decide you’d like to monitor or exclude some more files you can update /etc/tripwire/twpol.txt.  If you’re monitoring this file you’ll need to update the database as per the above section.  After that you can update the signed policy file (you’ll need your site and local keys for this).

sudo tripwire --update-policy /etc/tripwire/twpol.txt

As you can see tripwire can be an amazingly powerful tool in any security arsenal.  We use it as part of our maintenance plans and encourage others to do the same.

Feature image by Nathalie licensed CC BY 2.0.

The post Tripwire – How and Why appeared first on Dogsbody Technology.

We have one

Our status page status.dogsbody.com, has been running for quite a while. We suggest our customer use it as your first point of call when you spot something odd.  If you’ve got your own server(s) that we maintain, we’ll contact you directly if we start seeing issues.  The status page covers the below…

Support – methods we usually communicate with you

• Email
• Telephone
• Slack

Hosting – our shared hosting servers

• Indigo (our WordPress only hosting)
• Purple (our general purpose hosting)
• Violet (our cPanel hosting)

When we schedule maintenance or have issues we’ll update you via the status page.  If you are an Indigo or Purple customer and want to be notified of issues or maintenance go ahead and subscribe.

You want one

Having an (up to date) status page improves your users experience.  It gives them a quick way to find out what’s going on.  This means they’ll be have a better understanding and (usually) more tolerant of issues you’re already dealing with.

Having a status page is likely to cut down on the number of similar questions you get if you have an outage.  We’ve been really happy with the self-hosted open source software we’re using – Cachet.  We wanted to make sure our status page doesn’t go down at the same time as our other services.  So we’ve used a different server provider to our main infrastructure.  If you want to avoid worrying about that sort of thing, we’ve seen a lot of people are using statuspage.io and status.io.

Feature image background by Wolfgang.W. licensed CC BY 2.0.

The post What are Status Pages? appeared first on Dogsbody Technology.

A while ago we built our Slackbot, it’s a bit of fun, from providing inspirational quotes to tracking who’s made the most cups of tea. It’s well on it’s way to doing everything you never wanted.

The Problem

DogsBOT reacts to discussions regarding lunch. Initially this was with ? emoji. Since we don’t eat spaghetti every day I switched it to the ? emoji but this also became routine.  What we needed was a less predictable, more intelligent reaction.  My criteria for this was:

• Linked to the current weekly menu
• Some element of chance

A Solution

The canteen menu for the week is available as a PDF on the site intranet. It was an easy choice to use it as the source for DogsBOT’s lunch reactions.  As there are multiple options each day DogsBOT would be able to pick one for today at random.

With that decided we just needed to convert the PDF of the menu to text.  Given my bias to use python, a quick google indicated PyPDF2 would be the perfect tool. Unfortunately the menu turned out to be almost entirely images.  All I was able to pull out was the title and a bunch of meta data.  I needed to switch to an image recognition tool.

For the image recognition I wanted to run the code locally so ignored the various cloud solutions.  This still left a lot of choice of software.  I found this guide which did everything I was looking for using OpenCV (to clean up the image) and Tesseract (to pull out the text).

Now I’d got the text, I needed to work out which day the food would be served on. I was starting to run out of time so I set-up a quick loop.  It assumes each item is from the day of the week it most recently passed.  Happy that this was mostly working so I moved on to converting the food to emoji.  I did this by attempting to convert each word to an emoji and using the ones that were successful.  In hindsight I would have been better off selecting reactions based on shortnames in this data as Slack reactions use them rather than emoji.

Conclusion

The new improved lunch plugin mostly works.  I had fun playing with PyPDF2, ImageMagick, OpenCV and Tesseract as well as learning a little more about how Slack reactions work.

Tesseract is impressively good.   Given that I’d been planning to just pull the text out of the PDF I didn’t spend huge amount of time comparing it with the alternatives since it did everything I needed.  If spent more time on this I’d like to spend more time tweaking the OpenCV commands to work with different lower quality images.

More hackathons

Whether we’re fitting out a storm trooper or camping in a field we all love spending a day out doing something different.  For our next hackathon we’re planning to try and break in to our internal servers.  If you’ve got any hackathons lined up you think we’re missing out on let us know in the comments.

The post Hackathon – Image Recognition Slackbot appeared first on Dogsbody Technology.

Feature image by See1,Do1,Teach1 licensed CC BY 2.0.

The post PHP 5.6 will go end of life on 31 Dec 2018 appeared first on Dogsbody Technology.