David Yin's Blog

Migrating GitLab to another Server from Ubuntu 16.04 to Ubuntu 18.04

November 8, 2019
0 Comments

Because my old Gitlab is running on a Ubuntu 16.04 server. The latest LTS version of Ubuntu is already 18.04 for more than one year. I was thinking about to directly upgrade the server from 16.04 to 18.04.  There are still some chances it may fail and the server may crash. I can not afford to lose the data.

So the final decision is to migrate the Gitlab from the old server Ubuntu 16.04 to a new server with  Ubuntu 18.04.

This is how to do it.

First, I install the new server on my Proxmox VE environment with Ubuntu 18.04. 3CPU Cores and 4GB memory and 64GB storage. Meanwhile, the old server still uses Ubuntu 16.04. And the Gitlab version that I use is 12.4.2

On the Old Server

  1.  Shutdown Gitlab service 
    sudo gitlab-ctl stop unicorn
    sudo gitlab-ctl stop sidekiq
  2. Backup Gitlab on the old server 
    sudo gitlab-rake gitlab:bakcup:create
  3. Create a folder named gitlab-old 
    mkdir ~/gitlab-old
  4. Copy the backup file to the folder ~/gitlab-old 
    sudo cp ~/backups/xxxxxxxx_gitlab_backup.tar ~/gitlab-old
  5. Copy the Gitlab configuration files 
    sudo cp /etc/gitlab/gitlab.rb ~/gitlab-old
    sudo cp /etc/gitlab/gitlab-secrets.json ~/gitlab-old
    sudo cp -R /etc/gitlab/ssl ~/gitlab-old

Then all the important files are under folder ~/gitlab-old

My configuration of Gitlab changed the backup folder to ~/backups. The default path is /var/opt/gitlab/backups/

You can change it on the “Backup Settings” section in the file gitlab.rb

 

To verify the version of your gitlab installation, enter the following command.

sudo gitlab-rake gitlab:env:info

On the New Server

The server is a clean installation of Ubuntu 18.04. The only service is SSH  and postfix.

Before doing the migration, I transfer the whole folder ~/gitlab-old from the old server to this new server.

  1.  Add Gitlab source 
    curl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh | sudo bash
  2.  Install Gitlab 12.4.2 CE 
    sudo apt update
    sudo apt install gitlab-ce=12.4.2
  3. Copy the configuration files to folder /etc/gitlab 
    sudo cp ~/gitlab-old/gitlab* /etc/gitlab
  4. Copy the ssl folder to folder /etc/gitlab 
    sudo cp -R ~/gitlab-old/ssl /etc/gitlab
  5. Run Gitlab service for the first time 
    sudo gitlab-ctl reconfigure
  6. Shutdown the Gitlab services 
    sudo gitlab-ctl stop unicorn
    sudo gitlab-ctl stop sidekiq
  7. Copy the backup file to ~/backups then change the permission 
    sudo cp ~/gitlab-old/*.tar ~/backups
    sudo chown git:root ~/backups/*.tar
  8. Restore the Gitlab backup file 
    sudo gitlab-rake gitlab:backup:restore BACKUP=XXXXXX
  9. Restart Gitlab and check 
    sudo gitlab-ctl start
    sudo gitlab-rake gitlab:check SANITIZE=true

There are some notes for the above nine steps.

The backup file name looks like 1573175514_2019_11_07_12.4.2_gitlab_backup.tar. So the code in the step 8, BACKUP=1573175514_2019_11_07_12.4.2

The version of Gitlab I am using 12.4.2. I install the 12.4.2 on step 2. When I do the restore in step 8, it displays the error, the version does not match. The backup is 12.4.2, but the installation is 12.4.1. I have to do the apt update and apt upgrade to upgrade the Gitlab from 12.4.1 to 12.4.2. I am not sure where the problem came from. Just a reminder, double-check the version installed on the new server.

At the first step of adding the repo source, I always get the network error on my server. I believed it is the problem of guest OS on Proxmox VE. So I actually add the repo source and public key of gitlab source manually.

My current Gitlab information on the new server Ubuntu 18.04

davidyin@gitlab:~$ sudo gitlab-rake gitlab:env:info

System information
System: Ubuntu 18.04
Current User: git
Using RVM: no
Ruby Version: 2.6.3p62
Gem Version: 2.7.9
Bundler Version:1.17.3
Rake Version: 12.3.3
Redis Version: 3.2.12
Git Version: 2.22.0
Sidekiq Version:5.2.7
Go Version: unknown

GitLab information
Version: 12.4.2
Revision: 393a5bdafa2
Directory: /opt/gitlab/embedded/service/gitlab-rails
DB Adapter: PostgreSQL
DB Version: 10.9
URL: https://gitlab.g2soft.net
HTTP Clone URL: https://gitlab.g2soft.net/some-group/some-project.git
SSH Clone URL: git@gitlab.g2soft.net:some-group/some-project.git
Using LDAP: no
Using Omniauth: yes
Omniauth Providers:

GitLab Shell
Version: 10.2.0
Repository storage paths:
- default: /var/opt/gitlab/git-data/repositories
GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell
Git: /opt/gitlab/embedded/bin/git

At the end, I should give the credit to Didiet A. Pambudiono. Most of the steps above is based on this post.

Upgrade my Kindle Paperwhite 7th Gen. to v5.12.2

October 31, 2019
0 Comments

My Kindle Paperwhite 7th Generation was upgraded to 5.11.1.1 early this year. Late, I upgrade the firmware to 5.11.2. Now it is another upgrade available online.

Check the version before upgrading.

 

Firmware 5.11.2 What’s new

 

I touched the settings and settings again, to check the upgrade availability.

Upgrade notice

Continue reading “Upgrade my Kindle Paperwhite 7th Gen. to v5.12.2”

php script timeout on php7.2-fpm and Nginx Webserver

October 23, 2019
0 Comments

Here is the background.

  • It is a php script that is accessed through a browser.
  • Server: Ubuntu 18.04
  • Web server: Nginx 1.14
  • php: php7.2-fpm

 

The script always stoped on 60 seconds.

The Nginx error log is shown below:

2019/10/23 14:05:18 [error] 996#996: *950 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 135.0.172.188, server: fctu.win, request: “GET /go/do-something.php HTTP/2.0”, upstream: “fastcgi://unix:/var/run/php/php7.2-fpm.sock”, host: “mywebsite.com”

It is a timeout.  I knew it.

In the beginning, I am looking for the php timeout settings.

I tried to add max_execution_time to 360. It doesn’t help.

At last, I add a line into Nginx settings.

fastcgi_read_timeout 180;

Sample code below:

location ~ \.php$ {
    fastcgi_pass unix:/var/run/php7.2-fpm.sock;
    fastcgi_index index.php;
    fastcgi_param   SCRIPT_FILENAME $document_root$fastcgi_script_name;
    fastcgi_read_timeout 180;
    include fastcgi_params;
}

 

Rosary Miracle Story

October 21, 2019
0 Comments

Vienna, Austria, September 12, 1955: After World War II, Austria was divided between four countries: America, France, the United Kingdom, and Russia, which was still communist. The section of Austria controlled by the communists was the richest and included the city of Vienna. The Viennese were subject to all the atrocities and tyrannies of communism.

With all of his country’s problems weighing heavily on his heart, Capuchin Fr. Petrus Pavlicek made a pilgrimage to Mariazell, the principle Marian shrine in Austria. While deep in prayer before the miraculous image of Our Lady above the shrine’s high altar, he was told by an interior voice: “Do as I say and there will be peace.”

To obey this inspiration of Our Lady, Fr. Pavlicek founded the Holy Rosary Crusade of Reparation in 1947. This Crusade consisted of the Viennese faithful coming out of their homes in order to participate in a public Rosary procession in the streets of the city. The intentions of the Rosary were for the end of communism in their country and in the world. Father traveled throughout Austria with a statue of Our Lady of Fatima promoting the Rosary Crusade. At first, the processions were miniscule, but in time they grew to staggering proportions. The Prime Minister and other members of the Austrian government soon joined the ranks, along with all of the nation’s bishops.

In 1955, after eight years spreading the word about the Crusade throughout Austria, the Rosary processions would reach the size of half a million people, about one-tenth of the Austrian population. Finally,m through the help of Our Lady, the Soviet forces pulled out of Austria in October of 1955, leaving the country for good.

Each year on September 12th, the feast of the Holy Name of Mary, thousands gather in Vienna to thank the Mother of God for her intercession in freeing their country from communist domination.

— Canada Needs Our Lady.

The above paragraph is copied from the print of the rosary as a gift from my wife, Vivian.

Classification of files and tips to recover them

October 16, 2019
0 Comments

There are literally hundreds of computer files and formats that exist. It becomes impossible to understand and remember all of them. Hence, we are going to look at a broad classification of files, understand the major file types and how they are used in computers. Here is a breakdown of the file types.

A broad classification of files and file types

  1. System files

System files are files that are crucial for the functioning of a computer. These usually include hardware drivers, DLL files, configuration files and basically everything that includes the Windows registry.

Altering these files can cause severe damage; hence these files are usually kept hidden and are read-only. Hence it is a rare occurrence for users to accidentally delete or modify these files. Some examples include .bak, .dll, .dmp, .sys and .tmp.

Recovery tip #1: Lost system files could mean dysfunctional operating system. However recovering the system files will not repair the system files. Hence when you have deleted or lost system files the best way to get back your operating system to function normally would be to refresh or reinstall Windows operating system.

  1. Program files

Program files are the folder where all the third party applications are installed by default. For certain applications, it is mandatory that the applications be installed in this directory. But it is not always the case. You may also be given the choice to download applications in a folder of your choice.

Recovery tip #2: The program files directory is designed for an organized and predefined way of having all the required files in one place. The program when operated, folders of program files are accessed for opening the application. Hence reinstalling the program would be the best way to recover program files.

Continue reading “Classification of files and tips to recover them”

Calibre 4.1 Release

October 9, 2019
0 Comments

It is only four days after the release fo Calibre 4.0.

New features for Calibre 4.1

  • Viewer: Add an option to have a scrollbar (under Scrolling in the viewer preferences)
    Viewer: Allow showing the ‘position in book’, as was displayed in the top left of the viewer in calibre 3, in the header or footer of the viewer.
  • Viewer: Add options to control scrolling using the mouse in paged mode.
  • Viewer: Allow copying images by right clicking on them.
  • Viewer: Add a preference under Miscellaneous to allow only a single instance of the viewer
    Viewer: Add keyboard shortcuts to toggle between paged mode and flow mode and to quit
    Content server: Make the book size useable in custom book list templates.
  • Edit metadata: Use a border rather than background color to indicate correct title and author sort values. Works better with dark themes.
  • Kobo driver: Support for new firmware

It is a kinds of rush for this new release. more bug fixes on it.

New in calibre 4.0

October 5, 2019
0 Comments

Welcome back, calibre users. It has been two years since calibre 3.0. This time has been spent mostly in making the calibre Content server ever more capable as well as migrating calibre itself from Qt WebKit to Qt WebEngine, because the former is no longer maintained.

The Content server has gained the ability to Edit metadata, Add/remove books and even Convert books to and from all the formats calibre itself supports. It is now a full fledged interface to your calibre libraries.

The change to use Qt WebEngine has entailed a complete rewrite of various parts of calibre, including the E-book Viewer, PDF Output, Book details etc. Every attempt has been made to preserve features and functionality in a backwards compatible way, see below for a list of some known incompatibilities.

Wow, Calibre 4.0 is comming. I can not image how different it is from 3.x.

Calibre 3.48

 

I can wait to see it. Now download it 115MB. Install it. Then launch it.

Calibre 4.0

5 Essential Graphic Design Skills You Should Master

October 3, 2019
0 Comments

Maybe you’re a contract graphic designer doing small, one-off projects for local businesses. Or, an in-house professional responsible for overseeing all graphic creation in a large corporation. Regardless, you can always be improving your craft. 

Read on as we go over a few more advanced graphic design skills that will substantially improve your ability to communicate your message in the medium.

1. User Interface Design

At first glance, you might think this doesn’t apply to you. Perhaps your work focuses on something where ‘user interfaces’ seem worlds away from your final product. 

Consider this; regardless of what your final product is, the odds of it ending up within a user interface are virtually guaranteed. If you make a graphic for an article header for a local newspaper, for example, that graphic will likely end up in the online version of the publication.

And like it or not, that makes it a part of the user interface of the website. 

Knowing how to design with that type of presentation in mind already makes this a useful skill. Designing user interfaces directly, meanwhile, makes it substantially more worthwhile since the available pool of work for those who can do both is much larger.

Further, user interface creation is one of the more lucrative ways to apply the skills of a graphic designer. Per Indeed.com, user interface designers make, on average, more than double the income of a traditional graphic designer.

Continue reading “5 Essential Graphic Design Skills You Should Master”

How to Protect Your Smartphone from Online Threats

September 30, 2019
0 Comments

Every day, hundreds of people have their smartphone hacked, exposing important data like contacts and even credit card details to criminals that prey upon poorly protected phones.

Whenever your phone connects to the internet, it is at risk from hacking by data thieves that target phones in particular, as they contain nearly all of our important information such as passwords, emails and even the login details to online banking apps.

Fortunately, there are a few steps anyone can take to protect their phones from harm and recover their data if their smartphone is damaged or locked my malicious software.

Use a Virtual Private Network App When Connected to Public Wi-Fi

When you are using public Wi-Fi, such as when you are at a hotel or café, your phone is at risk of being hacked by someone else using that network. Some data thieves are known to connect to these networks regularly in order to steal credit card information from shoppers and tourists.

Virtual Private Networks encrypt and conceal your web traffic, even when using a public Wi-Fi network. These services can also be used from home to protect your online shopping and browsing there, giving you extra peace of mind.

Use an App to Securely Back Up All of Your Smartphone’s Data

Sometimes when a phone is hacked, it can suddenly become corrupted and unusable, trapping your files like your phone numbers and personal photographs on the device. There are apps available that can back up data to an online cloud service, but sometimes they can be missing important and more recent files.

If you have data on a damaged or corrupted smartphone, there are services like mobile data recovery by Secure Data Recovery that can recover data from your device, sometimes even if it has been deleted by a hacking program.

Use a Password Manager to Encrypt and Generate Your Passwords

Our passwords are gatekeepers to a lot of our sensitive information such as emails and bank accounts. Too many people rely on using the same password, or maybe two or three, in order to log in to all their different accounts.

Password managers use encryption algorithms to generate complex passwords for you, and store them in a ‘digital safe’ in an app on your phone and online on a secure server. The password manager will auto-complete passwords on all your accounts, or they can be copy-and-pasted from the app to the login form. This means you have only one password to remember, the one that logs you into your password manager, and you can let it safely and securely do the rest.

We keep a large amount of sensitive data on our phones, and not just banking details and contact information. Your phone probably has hundreds of photos of friends and family, as well as pictures from social media accounts. This information needs to be protected, not just from theft, but also from accidents and problems with a smartphone’s components. With a few apps and the help of professionals, anyone can protect the data on their phone, and recover it if the worst happens.

  • Archives