Site Updates and Feature Requests
- This topic has 7 replies, 1 voice, and was last updated 1 month ago by
.
-
Posts
-
This topic is for details of site updates such as new features, and also nerdy stuff such as software or server updates.
For more general news of new content added to the site, see the “What’s New?” section on the Huddersfield Exposed home page and also the news blog.
If there’s a new feature that you’d like to see added to the site or perhaps you don’t think that something is working properly, feel free to leave a reply!
Site Specs
main web server:
- Ubuntu 20.04 LTS running on Linode 2GB (London) + backup service
- Linode Object Storage (~100GB)
- Nginx 1.18.x
- Apache 2.4.x with ModSecurity & mod_perl
- PHP 7.4.x (FPM)
- MediaWiki 1.35.x (LTS) with CirrusSearch
- WordPress
- Omeka
database server:
- 3 x Debian 11 running on Linode 2GB (London)
- Galera Cluster (3 nodes) running MariaDB 10.6.x
search index server:
- Debian 11 running on Linode 4GB (London)
- ElasticSearch
- Redis
major config changes:
- 02/05/2020: moved ElasticSearch from the web server onto the database server (not ideal, as ES is a memory hog but needed to free up memory on the web server to cope with the huge increase in traffic since lockdown)
- 29/05/2020: changed the MediaWiki job queue to a continuous service
- 30/07/2020: moved ElasticSearch back onto the main web server as traffic levels have dropped slightly from the peak around late May 2020
- 15/08/2020: migrated database to new Ubuntu 20.04 server running MariaDB
- 18/08/2020: upgraded database server hardware and moved ElasticSearch onto it
- 08/12/2020: site upgraded to MediaWiki 1.35 (LTS)
- 30/12/2020: hardware upgrade to MySQL/ElasticSearch server (2GB to 8GB)
- 31/01/2021: site upgraded to MediaWiki 1.35.1 (LTS)
- 28/05/2021: site upgraded to MediaWiki 1.35.2 (LTS)
- 28/07/2021: site patched to MediaWiki 1.35.3 (LTS)
- 07/10/2021: ElasticSearch moved to separate 4GB server / database server reduced to 4GB
- 13/10/2021: site patched to MediaWiki 1.35.4 (LTS)
- 21/11/2021: memcached replaced by Redis
- 11/12/2021: ElasticSearch server patched against CVE-2021-44228
- 21/12/2021: migrated databases across to a 3-node Galera Cluster
- 30/12/2021: site patched to MediaWiki 1.35.5 (LTS)
- 30/12/2021: remove Old Maps UK and Pastscape from the “Details & Links” tab for locations as both sites are now sadly defunct
- 30/12/2001: fixed the Bench Mark Database link in the “Details & Links” tab to work with their new site
future plans:
- rewrite MW skin to resolve current JS loading issues
- integrate Omeka content directly into the wiki (so that the content appears in searches, etc) and just use Omeka as a back-end management tool
Hosting Costs
- 1 x Linode 2GB (web server) – $120 per year
- 1 x Linode 4GB (search index servers) – $240 per year
- 3 x Linode 2GB (3 node MariaDB Galera cluster) – $360 per year
- Linode Backups Service – $150 per year
- Linode Object Storage – $60 per year
- domain name -$25 per year
- email hosting – $15 per year
- TOTAL:
- USD: $970 + VAT
- UK (inc. VAT): approx £860 per year
Synonym searching is now enabled on Huddersfield Exposed. This is primarily being used to provide search across common variant spellings, such as:
- Hellawell / Helliwell / Hellwell
- Hinchliffe / Hinchcliff / Hinchcliffe / Hinchliff
- Lingards / Lingarths
- Linthwaite / Linfit
- Slaithwaite / Slawit
The initial emphasis is on adding synonyms to improve searching across the Holmfirth Flood Project content where spellings of names and places varies from one newspaper report to the next.
For example, a search for linfit hall will also match linthwaite hall…
Using quotation marks overrides the synonym searching, allowing you to focus on a specific spelling…
The current list of synonyms can be viewed here: https://huddersfield.exposed/wiki/HuddersfieldExposed:Synonyms
If there are any synonyms you’d like adding, please let me know.
The site was an early adopter of Linode’s Object Storage product and much of the multimedia content is stored “in the cloud”. At the time, the only option was to use the US-based storage.
As Linode now offer a European option and are able to support custom SSL certificates, I’ve begun migrating the bulk of the content across and it will be served from the domain storage.huddersfield.exposed. This should give slightly faster loading times for everyone in the UK.
The 1.34 branch of MediaWiki is now officially “end of life“.
Although there’s no immediate rush to move to 1.35 (LTS), I’ve decided it’d be better to carry out the upgrade before we begin loading the content of the newspaper OCR project as that will add over 43,000 new pages to the site.
I’ve carried out a successful test upgrade overnight, so the plan is to take the site offline for a short period tomorrow morning (Tuesday 8 December) to do the upgrade on the live server. If all goes well, it should only take about 15 minutes.
EDIT: the upgrade to 1.35 is now completed
The server hardware upgrades necessitated by the newspaper OCR project now means that the database server is the fastest server and has spare CPU capacity. Therefore, I’ve begun migrating the mod_perl API code from the main web server onto the database server. The API code was written to be fast but is now even quicker 🙂 In turn, this frees up capacity on the main web server.
The site is now upgraded to MediaWiki 1.35.2
The bad news is that a test upgrade to 1.36.0 failed and it looks like the skin used by the site will need to be rewritten from scratch due to it being written a couple of years ago using the now outdated MW Skinning Guide. Ho hum!
Since the 1.35 branch is a long term support (LTS) release, my current intention is to develop a new skin for the site and then migrate to the next stable LTS release at some point prior to September 2023.
To try and make the website more resilient, I’ve completed a migration of the databases from a single Linode 4GB server to a 3-node MariaDB Galera Cluster (where each node is a Linode 2GB server).
- You must be logged in to reply to this topic.