Mise à jour d’un serveur Kimsufi (OVH) depuis Debian 9 (Stretch) vers Debian 10 (Buster)

Il faut régulièrement penser à mettre à jour son serveur Kimsufi.

Je vais essayer d’expliquer brièvement les étapes à suivre pour cela.

  1. On va effectuer une mise à jour des paquets avec apt-get update && apt-get upgrade
  2. Il peut être intéressant de vérifier la version de son kernel afin d’avoir la dernière recommandée. On va vérifier la version actuelle avec uname -r qu’on peut comparer sur https://tracker.debian.org/pkg/linux. Si on a un noyau OVH, il est recommandé de passer sur une version classique (voir cette discussion). Ensuite, on fait:
    1
    2
    3
    apt-get install linux-image-amd64
    update-grub
    reboot

    Après le redémarrage, on peut vérifier la version du kernel avec uname -r

  3. Si vous avez une version personnalisée de MariaDB, vous pouvez vérifier que vous avez la dernière mise à jour pour Stretch
  4. On va sauvegarder les données :
    1
    mkdir /root/svg_special; cp -R /var/lib/dpkg /root/svg_special/; cp /var/lib/apt/extended_states /root/svg_special/; dpkg --get-selections "*" > /root/svg_special/dpkg_get_selection; cp -R /etc /root/svg_special/etc
  5. Ensuite il est conseillé d’utiliser screen pour pouvoir se reconnecter (avec screen -r) à en cas de déconnexion :
    1
    screen
  6. Le processus de mise à niveau décrit sur le site de Debian a été conçu pour des mises à niveau des systèmes « purs » sans paquet provenant d’autres sources. Pour une meilleure fiabilité du processus de mise à niveau, vous pouvez supprimer ces paquets du système avant de commencer la mise à niveau :
    1
    aptitude search '~i(!~ODebian)'
  7. On peut éventuellement purger les vieux paquets obsolètes. Pour cela on va d’abord les lister, puis on peut les purger si tout semble bon:
    1
    2
    aptitude search '~o'
    aptitude purge '~o'
  8. On peut lancer la commande dpkg --audit pour s’assurer que tout est bon avant la migration. On peut également taper dpkg --get-selections "*" | more et vérifier qu’aucun paquet n’est en on hold
  9. Maintenant il faut remplacer tous les “stretch” de /etc/apt/sources.list par des “buster” (on pourra par exemple utiliser sed -i 's/stretch/buster/g' /etc/apt/sources.list), ce qui va donner chez moi :
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    deb http://ftp.fr.debian.org/debian buster main non-free
     
    deb http://debian.mirrors.ovh.net/debian/ buster main
    deb-src http://debian.mirrors.ovh.net/debian/ buster main
     
    deb http://security.debian.org/ buster/updates main
    deb-src http://security.debian.org/ buster/updates main
     
    deb http://ftp.debian.org/debian buster-backports main
     
    deb http://debian.mirrors.ovh.net/debian buster-updates main
    deb-src http://debian.mirrors.ovh.net/debian buster-updates main

    On vérifiera aussi les autres fichiers qui peuvent se trouver dans /etc/apt/sources.list.d, en modifiant par exemple la source pour MariaDB.

  10. Il est recommandé d’utiliser le programme /usr/bin/script pour enregistrer une transcription de la session de mise à niveau. Ainsi, quand un problème survient, on a un enregistrement de ce qui s’est passé. Pour démarrer un enregistrement, taper :
    1
    script -t 2>~/upgrade-stretch.time -a ~/upgrade-stretch.script
  11. On passe aux choses sérieuses, en commençant par mettre à jour les listes des paquets :
    1
    apt-get update
  12. On va vérifier qu’on a la place suffisante (un message explicite apparait sinon) :
    1
    apt -o APT::Get::Trivial-Only=true full-upgrade
  13. On va maintenant faire une mise à jour minimale :
    1
    apt-get upgrade
  14. Et à partir de là le système va vous questionner… en général choisir l’option par défaut si vous ne savez pas quoi répondre
  15. Puis on continue avec
    apt full-upgrade

Cette dernière étape va durer un certain temps. Une fois terminé, vous pouvez redémarrer le serveur pour s’assurer que tout va bien.

Il est bien de vérifier que la version actuelle de PHP est correctement utilisée par Apache. Pour cela on vérifie la version avec:

php -v

Ensuite on regarde les versions de PHP disponibles dans les modules d’Apache :

ls -l /etc/apache2/mods-available/php*

Et on regarde celle activée :

ls -l /etc/apache2/mods-enabled/php*

Si la version ne correspond pas, alors on désactive la version actuellement gérée par Apache2 puis on active la dernière. Par exemple de PHP 7.0 à PHP 7.4 :

a2dismod php7.0
a2enmod php7.4

Et on redémarre Apache :

systemctl restart apache2

Il pourrait aussi être nécessaire d’installer php7.4-mysql :

apt-get install php7.4-mysql

Dans mon cas j’ai un serveur qui utilise Tomcat8 avec une vieille version de Java (“1.8.0_265”) contre la 1.11 qui a été installé pendant la mise à jour (on le voit avec java -v). La 1.8 n’est pas disponible pour Duster, donc j’ai suivi ces conseils :

apt-get update
apt-get install software-properties-common

Puis dans /etc/apt/sources.list.d/jdk8.list j’ai mis :
deb http://security.debian.org/debian-security stretch/updates main
Et ensuite :

apt-get update
apt-get install openjdk-8-jdk

Ensuite il faut modifier $JAVA_HOME… Pour cela on modifier d’abord /etc/environment en y mettant JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/ ainsi que /etc/default/tomcat8

Des erreurs sur PHPMyAdmin peuvent survenir dû au passage à PHP 7.4. Il est recommandé de mettre à jour PHPMyAdmin. On va d’abord vérifier qu’on a une source buster-backports dans /etc/apt/sources.list. Dans mon cas j’ai pu mettre à jour comme cela :

apt-get upgrade php-tcpdf php-twig
apt-get install -t buster-backports php-twig
apt-get install phpmyadmin


Une fois les erreurs corrigées, on va nettoyer tous les paquets avec :
1
apt-get autoremove

On arrête screen en faisant CTRL + A puis k.

Connectivity issue with Qualcomm QCA61x4A 802.11ac in 5GHz

I have a Dell computer with the wireless adaptater “Qualcomm QCA61x4A 802.11ac” which supports Wifi in 5GHz. However, I’ve recurrent deconnections when being connected to my 5GHz network, even if all my other devices don’t have any issues. It works well with the 2.4GHz network.

I’m on Windows 10 and after trying many different things, I found a workaround:

  1. Open a CMD window (by pressing Windows key + X, then select Windows PowerShell (Admin))
  2. Then type: netsh wlan set autoconfig enabled=no interface="Wi-Fi"
  3. You won’t see any WiFi network anymore from the Wifi icon, but it stops the issues with the connectivity

If you need to connect to a different network, or if you have restarted your computer, you may need to turn it on again with the below command:
netsh wlan set autoconfig enabled=yes interface="Wi-Fi"

Once connected, make sure to turn if off again.

Let’s Encrypt Certificate: how to remove a domain from a certname that contains several domains

My server manages several websites with different domains using Apache. The first time I used Let’s Encrypt I followed the default command which has created one certname for ALL my domains.

Now I want to remove just one domain from this certificate, and it becomes complicated to understand how to do it. The best solution is to create a new certificate for each of my domains, and then to delete the original certname.

Let’s say my certname is called www.example.com and it contains the below domains:

  • www.example.com
  • example.com
  • blog.example.com
  • other-example.com
  • www.other-example.com
  • my-other-domain.com
  • www.my-other-domain.com
  • api.test.com

The one I don’t need anymore is *.my-other-domain.com.

First, we create a certificate individually for each domain that we want to keep:

1
2
3
certbot --apache --cert-name example.com -d example.com,www.example.com,blog.example.com
certbot --apache --cert-name other-example.com -d other-example.com,www.other-example.com
certbot --apache --cert-name test.com -d api.test.com

--cert-name permits to give our own name to the certificate, and -d indicates which domains should be added to this certificate.

Then we can list all our certificates:

1
certbot certificates

Using the above command you can find the Certificate Path and now we can delete our original certificate:

1
certbot revoke --cert-path /etc/letsencrypt/live/www.example.com/fullchain.pem

You’re all set! All your domains should still have a correct certificate, and you revoked the ones you don’t need anymore.

Upgrade from MySQL (5.6.40 – Debian 9.12) to MariaDB

(Source)

A few steps:

1
2
3
sudo apt-get install software-properties-common dirmngr
wget -qO - https://mariadb.org/mariadb_release_signing_key.asc | sudo apt-key add -
nano /etc/apt/sources.list.d/mariadb.list

In mariadb.list we add the two below lines:

1
2
deb [arch=amd64,i386,ppc64el] http://mirror.23media.de/mariadb/repo/10.4/debian stretch main
deb-src http://mirror.23media.de/mariadb/repo/10.4/debian stretch main

Then:

1
2
apt-get update
apt-get install mariadb-server

chrome-search://local-ntp/local-ntp.html makes Chrome unresponsive

Sometimes, when I open a new tab in Chrome, it first tries to get chrome-search://local-ntp/local-ntp.html and while it’s doing it, I cannot enter any URL. Eventually it will popup a message asking me if I want to wait or exit the page…

The way I found to fix it is to go to the page chrome://flags/#use-google-local-ntp, then I disabled “Enable using the Google local NTP”

Our system has detected an unusual rate of unsolicited mail originating from your IP address

Gmail could report back this error message: “Our system has detected an unusual rate of unsolicited mail originating from your IP address”

If your DKIM, SPF and DMARC are all set and your emails are correctly identified, then you may want to check the both links:

Sharepoint REST API to index a column or delete a column and more…

Due to the 5,000 items threshold limitation it can become very frustrating to administrate Sharepoint. For example, if your list has more than 5,000 items, you cannot add an index, delete a lookup/people column, delete the list or the website, and more !

Hopefully some of the operations can be done with REST API, and in my organization the threshold limitation is removed during the night (1am to 4am). By combining a schedule service (like node-schedule) with my library SharepointPlus I can run this admin tasks during the night while my virtual computer is running.

To delete a column

1
2
3
4
5
6
7
8
$SP().ajax({
  url:"https://company.com/sharepoint/team/sales/_api/web/lists/getbytitle('My List')/fields/getbytitle('Column To Delete')",
  method: "POST",
  headers: {
    "IF-MATCH": "*",
    "X-HTTP-Method": "DELETE"
  }
});

To index a column

1
2
3
4
5
6
7
8
9
10
11
12
$SP().ajax({
  url:"https://company.com/sharepoint/team/sales/_api/web/lists/getbytitle('My List')/fields/getbytitle('Column to Index')",
  method: "POST",
  body:JSON.stringify({
    "__metadata":{ type: "SP.Field" },
    "Indexed":true
  }),
  headers: {
    "IF-MATCH": "*",
    "X-HTTP-Method": "MERGE"
  }
})

To delete a list

1
2
3
4
5
6
7
8
$SP().ajax({
  method: "POST",
  headers: {
    "IF-MATCH": "*",
    "X-HTTP-Method": "DELETE"
  }
})

To delete a list

1
2
3
4
5
6
7
8
$SP().ajax({
  method: "POST",
  headers: {
    "IF-MATCH": "*",
    "X-HTTP-Method": "DELETE"
  }
})

To delete a site

1
2
3
4
5
6
7
$SP().ajax({
  method: "POST",
  headers: {
    "X-HTTP-Method": "DELETE"
  }
})

SharePoint 2013 Search REST API options

Finding the documentation for the Search REST API of Sharepoint is difficult…. There is a very old blog post that I’m going to replicate here to give more visibility:

 

Location of the Search Rest service

The Search REST service is located at the following URI: http://host/site/_api/search

 

The Search REST API

The Search REST service exposed by the URI specified above provides the following methods or entry points:

 

 

query

http://host/site/_api/search/query

Supports HTTP Get and is used to retrieve search results.

postquery

http://host/site/_api/search/postquery

Supports HTTP POST and is used to retrieve search results. This method can be used to overcome URI length restrictions when using the HTTP GET based method “query“.

suggest

http://host/site/_api/search/suggest

Supports HTTP GET and is used to retrieve query suggestions.

 

Query Parameters

The following table lists the query parameters available for the query entry point:

 

Name

Description

Type

Default

querytext

The text of the search query.

String – Maximum query text length allowed by the Keyword Query OM is 4096 characters.

Empty

http://host/site/_api/search/query?querytext=’search is cool’

querytemplate

The query template text.

String

Empty

 

enableinterleaving

Specifies if the result sets which are generated by executing query rule actions to add result  block should be mixed with the result set for the original query.

bool

True

http://host/site/_api/search/query?querytext=’terms’&enableinterleaving=false

sourceid

Specifies the unique identifier of the Result Source to use for executing the search query.

String

Empty

http://host/site/_api/search/query?querytext=’term’&sourceid=’8413cd39-2156-4e00-b54d-11efd9abdb89′

rankingmodelid

The GUID of the Rank Model to be used for this search query.

String

Empty

 

startrow

A zero-based index of the first search result to be returned.

Integer [ >= 0]

0

http://host/site/_api/search/query?querytext=’terms’&startrow=11

rowlimit

The maximum number of search results to be returned, starting at the index specified in  startrow.

Integer [ >= 0]

10

http://host/site/_api/search/query?querytext=’terms’&startrow=11&rowlimit=3

rowsperpage

The number of results to return per page.

Integer [ >= 0]

0

 

selectproperties

Specifies the list of managed properties to return for each search result item. For a managed property to be returned; the Retrievable flag must be set to “true” in the Search Schema.

String

Empty

http://host/site/_api/search/query?querytext=’terms’&selectproperties=’Path,Url,Title,Author’

culture

Specifies the identifier of the language culture of the search query. If present, the value must be a valid LCID of a culture name. A list of LCIDs is available here.

Integer

-1

http://host/site/_api/search/query?querytext=’terms’&culture=1044

refiners

Specifies a list of refiners to return as a comma-separated list of strings.

String

Empty

http://host/site/_api/search/query?querytext=’terms’&refiners=’companies,contentclass,ContentType,FileExtension,FileType’

refinementfilters

The list of refinement tokens for drilldown into search results. Refinement tokens are returned as part of the RefinementResults table for the search query.

String

Empty

http://host/site/_api/search/query?querytext=’sharepoint’&refiners=’filetype’&refinementfilters=’filetype:equals(“pptx”)’

hiddenconstraints

Specifies additional query terms that will be appended to the query.

String

Empty

http://host/site/_api/search/query?querytext=’terms’&hiddenconstraints=’powerpoint’

sortlist

Specifies the list of properties to sort the search results by.

String

Empty

http://host/site/_api/search/query?querytext=’terms’&sortlist=’rank:ascending’

enablestemming

Specifies whether stemming is enabled for the query.

bool

True

http://host/site/_api/search/query?querytext=’terms’&enablestemming=false

trimduplicates

Specifies whether duplicate items should be removed from search results. This property can also
 be used to collapse hits in the result set.

bool

True

http://host/site/_api/search/query?querytext=’terms’&trimduplicates=false

trimduplicatesincludeid

Specifies the value associated with a collapse group, typically used when a user clicks the duplicates link of an item with duplicates.

long

0L

http://host/site/_api/search/query?querytext=’terms’&trimduplicates=true&trimduplicatesincludeid=47

timeout

Specifies the amount of time, in milliseconds, before the query request times out.

Integer

30000

http://host/site/_api/search/query?querytext=’terms’&timeout=50000

enablenicknames

Specifies whether the exact terms in the search query are used to find matches, or if nicknames
 are used as well.

bool

False

http://host/site/_api/search/query?querytext=’terms’&enablenicknames=true

enablephonetic

Specifies whether the phonetic forms of the query terms are used to find matches.

bool

False

http://host/site/_api/search/query?querytext=’terms’&enablephonetic=true

enablefql

Specifies whether the query string is according to the FAST Query Language (FQL) syntax.

Bool

False

http://host/site/_api/search/query?querytext=’terms’&enablefql=true

hithighlightedproperties

Specifies the list of properties to include in the HitHighlightedProperties for each result item.

String

Empty

http://host/site/_api/search/query?querytext=’terms’&hithighlightedproperties=’Title,Author’

bypassresulttypes

Specifies if the search result item type should be returned for the query results.

bool

False

 

processbestbets

Specifies if the search promoted results should be returned, if available, as a result set.

bool

True

 

clienttype

Name of a client which issued query.

String

Empty

 

personalizationdata

Gets or sets the unique identifier (GUID) for the current user who submitted the search query.

String

Empty

 

resultsurl

Specifies the URL for the page where the search results are going to be displayed.

String

Empty

 

querytag

Any custom tags to be used to identify the query. Multiple tags are separated by semicolons.

String

Empty

 

enablequeryrules

Specifies if Query Rules are turned on for this query.

bool

True

http://host/site/_api/search/query?querytext=’terms’&enablequeryrules=false

enablesorting

Indicates whether results should be sorted.

bool

True

http://host/site/_api/search/query?querytext=’terms’&enablesorting=false

 

 

Suggest Parameters

The following table lists the query parameters available from the suggest entry point:

  

Name

Description

Type

Default

querytext

The text of the search query.

String

Empty

http://host/site/_api/search/suggest?querytext=’nad’

inumberofquerysuggestions

The number of query suggestions to retrieve.

Integer [ > 0]

5

http://host/site/_api/search/suggest?querytext=’nad’&inumberofquerysuggestions=6

inumberofresultsuggestions

The number of personal results to retrieve.

Integer [ > 0]

5

http://host/site/_api/search/suggest?querytext=’nad’&inumberofresultsuggestions=6

fprequerysuggestions

Specifies whether to retrieve pre-query suggestions or post-query suggestions.

bool

False

http://host/site/_api/search/suggest?querytext=’nad’&fprequerysuggestions=true

fhithighlighting

Specifies whether the query suggestions should be highlighted. If “true”, the terms
 in the returned query suggestions that match terms in the specified query are  surrounded with <B> and </B> HTML tags.

bool

True

http://host/site/_api/search/suggest?querytext=’nad’&fhithighlighting=true

fcapitalizefirstletters

Specifies whether to capitalize first letters in each term in query suggestions.

bool

False

http://host/site/_api/search/suggest?querytext=’nad’&fcapitalizefirstletters=true

showpeoplenamesuggestions

Specifies if people names should be included in query suggestions.

bool

False

http://host/site/_api/search/suggest?querytext=’nad’&showpeoplenamesuggestions=true

culture

Specifies the identifier of the language culture of the search query. If present, the value must be a valid LCID of a culture name. A nice list of LCIDs is available here.

Integer

-1

http://host/site/_api/search/suggest?querytext=’nad’&culture=1044

 

 

 References:

Examples

A postquery request will look like:

POST http://localhost/_api/search/postquery HTTP/1.1
Accept: application/json;odata=verbose
Content-Type: application/json;odata=verbose
Host: localhost
Content-Length: 419

{

 'request': {
     'Querytext': 'sharepoint',
     'StartRow':1,
     'RowLimit':8,
     'SelectProperties': {
          'results': ['Title','ContentSource','DisplayAuthor']
     },
     'TrimDuplicates':true,
     'Refiners':'companies,contentclass,ContentType(filter=7/0/*),FileExtension(filter=7/0/*),FileType(filter=6/0/*)',
     'RefinementFilters': {'results': ['filetype:equals("pptx")'] }
  }
}