Easy virtual machine (VM)
Introduction
This documentation is for virtual machines (VM) provided by Easy Software for Clients who bought Server solution. It is valid for VMs generated between March 1st 2022 and September 2024 (without embedded Docker).
It may also serve as inspiration of your own server environment, but in such case there are no guarantees due to the variety of possible components and technologies you may be using.
The VM is configured just for small sites (cca 10 simultanously working users). VM is almost production ready but REQUIRES experienced administrator to install VM in your environment, set up backuping, monitoring, networking, security policies etc. Easy Software can not take responsibility for running of this VM in your dedicated environment, the VM is an example how the application can be hosted.
This README addresses "Application", which can be substituted by either "Easy Project" or "Easy Redmine".
First steps
Because the VM was generated at a certain point in time, it is possible that some components may have already released some patches or security fixes by the time you are deploying it.
That is why you should always start with a backup followed by update of the OS. It will install the latest security fixes and update possibly outdated components.
VM description
OS is Debian in 64-bit edition.
The important defaults are:
- OS is available by SSH via port 22
- Firewall is not set (it is up to system administrators)
- The disk is ~100 GB
- /boot 512M
- /30G
- /home 60G
- swap 4G
- to increase disk, use the available LVM
- Application listens on HTTPS but with self-signed certificate, so you have to replace it ASAP (more details below)
- Application is running in /home/easy/[application_name]/current.
Linux login:
- username: easy
- password: e4syPwd-
Please change it ASAP. To do so run command passwd in terminal
Mysql login:
Simply use command mysql your_app_name (all dots or dashes in application name should be replaced with underlines. Example: your-app-name should be your_app_name)
Application login:
Default login (in case of "clean" database) is:
- username: admin
- password: admin
The default login for the VM with the demo data is:
- username: manager
- password: easy848
In case of database from your Cloud application, passwords remain as you had them.
Structure of the application's home
.
├── backups # backup script save data there
└── $APP_NAME
├── application.pid
├── application.sock # nginx uses this socket
├── config
│ ├── # pregenerated application configurations
│ ├── # (database, mailing, secrets, app server, etc...)
│ └── ...
├── current
│ ├── config # all application configurations
│ │ ├── configuration.yml # set up outgoing mail configuration here
│ │ ├── database.yml # database access configured here
│ │ └── ...
│ ├── files # persistent application data
│ ├── log # application logs
│ ├── public # static files
│ └── version # application version
└── public_html -> /home/easy/$APP_NAME/current
Restarting the application is simply via "systemctl restart puma@[application_name]" as sudo
Customer responsibilities
The VM runs on the latest recomended version of Debian. The system requires common maintenance like:
- regular updates
- resource monitoring (free space, memory usage, load, ...)
- data backups to an independent disk
- security policies
Updates
Do security updates of the system frequently. We recommend to do it at least once per week. Before update please be sure that latest backup is available and working.
Official documentation how to run only security updates https://wiki.debian.org/UnattendedUpgrades
Resource monitoring
Easy Software has good experience with Prometheus. Therefore, the node_exporter is a part of the VM since the 1.3. 2022.
The official very basic documentation for Node Exporter and Prometheus is at https://prometheus.io/docs/guides/node-exporter/ The best quickstart grafana dashboard is at https://grafana.com/grafana/dashboards/1860
Depending on your usage you should tune the VM resource settings, for example increase amount of RAM or number of cores provided to VM. All these configurations should be regularly done by your virtual environment administrator.
Node exporter listens on port 9100.
Disable node_exporter
node_exporter could by disabled by running:
systemctl stop node_exporter.service
systemctl disable node_exporter.service
systemctl mask node_exporter.service
It prevents to start node exporter after next reboot...
If you need full environment support from Easy Software (security and performance optimizations, backuping, monitoring, etc), we recommend to transfer to our Cloud solutions.
Data backups
The preconfigured backuping that is present in the VM is a working example, that needs to be fit to your infrastructure and has to be monitored. Please keep in mind, that a proper backup should be:
- checked after backup process finished (check at least the size of the backup)
- kept in a safe place
- distant from the running environment (at least in a different city)
- restored once in a while to test your ability to perform recovery
The application stores data in two places, so don't forget to backup both of them.
DB
Database backup can be very easily done incorrectly. The best and the most secure way how to do backups of mysql database is via database dump as you can see in the example script. It makes version independent backups, that could be restored whole or allows you to get only part of the old data.
Example:
mysqldump --add-drop-tables --routines --triggers --flush-logs $DATABASE_NAME > path/to/file_where_db_will_be_stored.sql
Copying files from /var/lib/mysql is an INCORRECT way of saving DB backups.
Files
Every uploaded file/attachment in the application is stored in /home/easy/$APP_NAME/current/files. The whole content of this directory should be backed up. In this case, you can simply copy the contents.
Security policies
The VM contains SSH keys for Easy Software engineers in home/easy/.ssh/authorized_keys. Also, the rules for the iptables firewall in /etc/iptables allow access of an Easy Software IP address.
Both of these policies are meant for eventual remote support by our staff on your request. The access policies are fully under your control.
Automatic tasks
All automatic tasks are scheduled by CRON and Sidekiq, which starts scheduled jobs every 5m. We recommend don't to run these jobs in shorter intervals due to their high performance requirements. To make any changes, you can edit them by running command below under user easy
crontab -e
Detailed documentation for cron
https://www.easyredmine.com/resources/redmine-installation/common-troubles/cron-not-working-helpdesk-emails-alerts-repeating-tasks-etc-are-not-automatically-processed
Migrate data to this VM
How to import older Application or Redmine data (SOURCE) into VM (TARGET). This works in most cases, but this procedure may not work in case you use 3rd party plugins in your source Redmine instance.
1. from your (SOURCE) create MySQL dump file:
mysqldump --opt easy > backup.sql
2. transfer backup.sql into (TARGET)
3. go to Application folder on (TARGET)
cd /home/easy/[application_name]/current
4. recreate (TARGET) database
bundle exec rake db:drop db:create RAILS_ENV=production DISABLE_DATABASE_ENVIRONMENT_CHECK=1
5. import backup.sql
mysql [db_name] < backup.sql
6. transfer attachments and other files from (SOURCE)[redmine]/files into (TARGET)/home/easy/[application_name]/current/files
7. run:
bundle exec rake easyproject:install RAILS_ENV=production
8. restart application
sudo systemctl restart puma@[application_name]
Troubleshooting
Missing sidekiq
On some of the machines sidekiq@.service file is not added. Please check /etc/systemd/system folder if sidekiq@.service file is presented. If not - please add it. It should contain the next:
[Unit]
Description=SIDEKIQ service for %i
After=redis.service
PartOf=puma@%i.service unicorn@%i.service
[Service]
Type=simple
PIDFile=/home/easy/%i/sidekiq.pid
Environment=RAILS_ENV=production
Environment=MALLOC_ARENA_MAX=2
User=easy
Group=www-data
WorkingDirectory=/home/easy/%i/public_html
ExecStart=/usr/local/rvm/bin/rvm default do bundle exec sidekiq -e production
ExecReload=/bin/kill -s TSTP $MAINPID
Restart=always
[Install]
WantedBy=multi-user.target puma@%i.service unicorn@%i.service
After file is created run
sudo systemctl daemon-reload
restart puma
MYSQL Backups are targeted to wrong database
To fix this issue please open file /usr/local/bin/mysql-backups.sh and replace DB name "easy" to app DB (usually it's the same as app name but all dashes and dots should be replaced with underlines)