Upgrade from MySQL (5.6.40 – Debian 9.12) to MariaDB

(Source)

A few steps:

sudo apt-get install software-properties-common dirmngr
wget -qO - https://mariadb.org/mariadb_release_signing_key.asc | sudo apt-key add -
nano /etc/apt/sources.list.d/mariadb.list

In mariadb.list we add the two below lines:

deb [arch=amd64,i386,ppc64el] http://mirror.23media.de/mariadb/repo/10.4/debian stretch main
deb-src http://mirror.23media.de/mariadb/repo/10.4/debian stretch main

Then:

apt-get update
apt-get install mariadb-server

Sharepoint error message « Invalid URL argument » with WebPartPages

For me the issue was this part of the XML request: <AddWebPart xmlns='http://microsoft.com/sharepoint/webpartpages/'> (with a / at the end of the url) instead of <AddWebPart xmlns='http://microsoft.com/sharepoint/webpartpages'>.

chrome-search://local-ntp/local-ntp.html makes Chrome unresponsive

Sometimes, when I open a new tab in Chrome, it first tries to get chrome-search://local-ntp/local-ntp.html and while it’s doing it, I cannot enter any URL. Eventually it will popup a message asking me if I want to wait or exit the page…

The way I found to fix it is to go to the page chrome://flags/#use-google-local-ntp, then I disabled « Enable using the Google local NTP »

Our system has detected an unusual rate of unsolicited mail originating from your IP address

Gmail could report back this error message: « Our system has detected an unusual rate of unsolicited mail originating from your IP address »

If your DKIM, SPF and DMARC are all set and your emails are correctly identified, then you may want to check the both links:

Sharepoint REST API to index a column or delete a column and more…

Due to the 5,000 items threshold limitation it can become very frustrating to administrate Sharepoint. For example, if your list has more than 5,000 items, you cannot add an index, delete a lookup/people column, delete the list or the website, and more !

Hopefully some of the operations can be done with REST API, and in my organization the threshold limitation is removed during the night (1am to 4am). By combining a schedule service (like node-schedule) with my library SharepointPlus I can run this admin tasks during the night while my virtual computer is running.

To delete a column

$SP().ajax({
  url:"https://company.com/sharepoint/team/sales/_api/web/lists/getbytitle('My List')/fields/getbytitle('Column To Delete')",
  method: "POST",
  headers: {
    "IF-MATCH": "*",
    "X-HTTP-Method": "DELETE"
  }
});

To index a column

$SP().ajax({
  url:"https://company.com/sharepoint/team/sales/_api/web/lists/getbytitle('My List')/fields/getbytitle('Column to Index')",
  method: "POST",
  body:JSON.stringify({
    "__metadata":{ type: "SP.Field" },
    "Indexed":true
  }),
  headers: {
    "IF-MATCH": "*",
    "X-HTTP-Method": "MERGE"
  }
})

To delete a list

$SP().ajax({
  url:"https://company.com/sharepoint/team/sales/_api/web/lists/getbytitle('List To Delete')",
  method: "POST",
  headers: {
    "IF-MATCH": "*",
    "X-HTTP-Method": "DELETE"
  }
})

To delete a list

$SP().ajax({
  url:"https://company.com/sharepoint/team/sales/_api/web/lists/getbytitle('List To Delete')",
  method: "POST",
  headers: {
    "IF-MATCH": "*",
    "X-HTTP-Method": "DELETE"
  }
})

To delete a site

$SP().ajax({
  url:"https://company.com/sharepoint/team/sales/website_to_delete/_api/web/",
  method: "POST",
  headers: {
    "X-HTTP-Method": "DELETE"
  }
})

SharePoint 2013 Search REST API options

Finding the documentation for the Search REST API of Sharepoint is difficult…. There is a very old blog post that I’m going to replicate here to give more visibility:

 

Location of the Search Rest service

The Search REST service is located at the following URI: http://host/site/_api/search

 

The Search REST API

The Search REST service exposed by the URI specified above provides the following methods or entry points:

 

 

query

http://host/site/_api/search/query

Supports HTTP Get and is used to retrieve search results.

postquery

http://host/site/_api/search/postquery

Supports HTTP POST and is used to retrieve search results. This method can be used to overcome URI length restrictions when using the HTTP GET based method « query« .

suggest

http://host/site/_api/search/suggest

Supports HTTP GET and is used to retrieve query suggestions.

 

Query Parameters

The following table lists the query parameters available for the query entry point:

 

Name

Description

Type

Default

querytext

The text of the search query.

String – Maximum query text length allowed by the Keyword Query OM is 4096 characters.

Empty

http://host/site/_api/search/query?querytext=’search is cool’

querytemplate

The query template text.

String

Empty

 

enableinterleaving

Specifies if the result sets which are generated by executing query rule actions to add result  block should be mixed with the result set for the original query.

bool

True

http://host/site/_api/search/query?querytext=’terms’&enableinterleaving=false

sourceid

Specifies the unique identifier of the Result Source to use for executing the search query.

String

Empty

http://host/site/_api/search/query?querytext=’term’&sourceid=’8413cd39-2156-4e00-b54d-11efd9abdb89′

rankingmodelid

The GUID of the Rank Model to be used for this search query.

String

Empty

 

startrow

A zero-based index of the first search result to be returned.

Integer [ >= 0]

0

http://host/site/_api/search/query?querytext=’terms’&startrow=11

rowlimit

The maximum number of search results to be returned, starting at the index specified in  startrow.

Integer [ >= 0]

10

http://host/site/_api/search/query?querytext=’terms’&startrow=11&rowlimit=3

rowsperpage

The number of results to return per page.

Integer [ >= 0]

0

 

selectproperties

Specifies the list of managed properties to return for each search result item. For a managed property to be returned; the Retrievable flag must be set to « true » in the Search Schema.

String

Empty

http://host/site/_api/search/query?querytext=’terms’&selectproperties=’Path,Url,Title,Author’

culture

Specifies the identifier of the language culture of the search query. If present, the value must be a valid LCID of a culture name. A list of LCIDs is available here.

Integer

-1

http://host/site/_api/search/query?querytext=’terms’&culture=1044

refiners

Specifies a list of refiners to return as a comma-separated list of strings.

String

Empty

http://host/site/_api/search/query?querytext=’terms’&refiners=’companies,contentclass,ContentType,FileExtension,FileType’

refinementfilters

The list of refinement tokens for drilldown into search results. Refinement tokens are returned as part of the RefinementResults table for the search query.

String

Empty

http://host/site/_api/search/query?querytext=’sharepoint’&refiners=’filetype’&refinementfilters=’filetype:equals(« pptx »)’

hiddenconstraints

Specifies additional query terms that will be appended to the query.

String

Empty

http://host/site/_api/search/query?querytext=’terms’&hiddenconstraints=’powerpoint’

sortlist

Specifies the list of properties to sort the search results by.

String

Empty

http://host/site/_api/search/query?querytext=’terms’&sortlist=’rank:ascending’

enablestemming

Specifies whether stemming is enabled for the query.

bool

True

http://host/site/_api/search/query?querytext=’terms’&enablestemming=false

trimduplicates

Specifies whether duplicate items should be removed from search results. This property can also
 be used to collapse hits in the result set.

bool

True

http://host/site/_api/search/query?querytext=’terms’&trimduplicates=false

trimduplicatesincludeid

Specifies the value associated with a collapse group, typically used when a user clicks the duplicates link of an item with duplicates.

long

0L

http://host/site/_api/search/query?querytext=’terms’&trimduplicates=true&trimduplicatesincludeid=47

timeout

Specifies the amount of time, in milliseconds, before the query request times out.

Integer

30000

http://host/site/_api/search/query?querytext=’terms’&timeout=50000

enablenicknames

Specifies whether the exact terms in the search query are used to find matches, or if nicknames
 are used as well.

bool

False

http://host/site/_api/search/query?querytext=’terms’&enablenicknames=true

enablephonetic

Specifies whether the phonetic forms of the query terms are used to find matches.

bool

False

http://host/site/_api/search/query?querytext=’terms’&enablephonetic=true

enablefql

Specifies whether the query string is according to the FAST Query Language (FQL) syntax.

Bool

False

http://host/site/_api/search/query?querytext=’terms’&enablefql=true

hithighlightedproperties

Specifies the list of properties to include in the HitHighlightedProperties for each result item.

String

Empty

http://host/site/_api/search/query?querytext=’terms’&hithighlightedproperties=’Title,Author’

bypassresulttypes

Specifies if the search result item type should be returned for the query results.

bool

False

 

processbestbets

Specifies if the search promoted results should be returned, if available, as a result set.

bool

True

 

clienttype

Name of a client which issued query.

String

Empty

 

personalizationdata

Gets or sets the unique identifier (GUID) for the current user who submitted the search query.

String

Empty

 

resultsurl

Specifies the URL for the page where the search results are going to be displayed.

String

Empty

 

querytag

Any custom tags to be used to identify the query. Multiple tags are separated by semicolons.

String

Empty

 

enablequeryrules

Specifies if Query Rules are turned on for this query.

bool

True

http://host/site/_api/search/query?querytext=’terms’&enablequeryrules=false

enablesorting

Indicates whether results should be sorted.

bool

True

http://host/site/_api/search/query?querytext=’terms’&enablesorting=false

 

 

Suggest Parameters

The following table lists the query parameters available from the suggest entry point:

  

Name

Description

Type

Default

querytext

The text of the search query.

String

Empty

http://host/site/_api/search/suggest?querytext=’nad’

inumberofquerysuggestions

The number of query suggestions to retrieve.

Integer [ > 0]

5

http://host/site/_api/search/suggest?querytext=’nad’&inumberofquerysuggestions=6

inumberofresultsuggestions

The number of personal results to retrieve.

Integer [ > 0]

5

http://host/site/_api/search/suggest?querytext=’nad’&inumberofresultsuggestions=6

fprequerysuggestions

Specifies whether to retrieve pre-query suggestions or post-query suggestions.

bool

False

http://host/site/_api/search/suggest?querytext=’nad’&fprequerysuggestions=true

fhithighlighting

Specifies whether the query suggestions should be highlighted. If « true », the terms
 in the returned query suggestions that match terms in the specified query are  surrounded with <B> and </B> HTML tags.

bool

True

http://host/site/_api/search/suggest?querytext=’nad’&fhithighlighting=true

fcapitalizefirstletters

Specifies whether to capitalize first letters in each term in query suggestions.

bool

False

http://host/site/_api/search/suggest?querytext=’nad’&fcapitalizefirstletters=true

showpeoplenamesuggestions

Specifies if people names should be included in query suggestions.

bool

False

http://host/site/_api/search/suggest?querytext=’nad’&showpeoplenamesuggestions=true

culture

Specifies the identifier of the language culture of the search query. If present, the value must be a valid LCID of a culture name. A nice list of LCIDs is available here.

Integer

-1

http://host/site/_api/search/suggest?querytext=’nad’&culture=1044

 

 

 References:

Examples

A postquery request will look like:

POST http://localhost/_api/search/postquery HTTP/1.1
Accept: application/json;odata=verbose
Content-Type: application/json;odata=verbose
Host: localhost
Content-Length: 419

{

 'request': {
     'Querytext': 'sharepoint',
     'StartRow':1,
     'RowLimit':8,
     'SelectProperties': {
          'results': ['Title','ContentSource','DisplayAuthor']
     },
     'TrimDuplicates':true,
     'Refiners':'companies,contentclass,ContentType(filter=7/0/*),FileExtension(filter=7/0/*),FileType(filter=6/0/*)',
     'RefinementFilters': {'results': ['filetype:equals("pptx")'] }
  }
}

exim4 : « remote_smtp: message is too big (transport limit = 1) »

After diging around for an exim4 problem (remote_smtp: message is too big (transport limit = 1)), I found the answer in this post:

Add « IGNORE_SMTP_LINE_LENGTH_LIMIT=1 » in the exim4 config file (/etc/exim4/update-exim4.conf.conf), then restart exim4

SP.Utilities.Utility.SendEmail returns « The e-mail message cannot be sent. Make sure the e-mail has a valid recipient. »

With Sharepoint you can use SP.Utilities.Utility.SendEmail to send email in JavaScript, however you could receive the below error:

« The e-mail message cannot be sent. Make sure the e-mail has a valid recipient. »

After searching the web, I found the solution: instead of using an email address for the recipients you have to use the userDomain with « i:0#.w| ».

For example, if the user is John Doe, and his email is john_doe@company.com, then you should use something like i:0#.w|domain\\john_doe.

mysqladmin with MariaDB returns « Access denied for user ‘root’@’localhost’ (using password: NO) »

When I want to reload my MariaDB server, I receive the below error:

/usr/bin/mysqladmin: connect to server at ‘localhost’ failed
error: ‘Access denied for user ‘root’@’localhost’ (using password: NO)’

The solution is to provide the root’s password in the file `/etc/mysql/debian.cnf` and then the error is gone!

Mise à jour d’un serveur Kimsufi (OVH) depuis Debian 8 (Jessie) vers Debian 9 (Stretch)

Il faut régulièrement penser à mettre à jour son serveur Kimsufi.

Je vais essayer d’expliquer brièvement les étapes à suivre pour cela.

  1. On va d’abord sauvegarder les données :
    mkdir /root/svg_special; cp -R /var/lib/dpkg /root/svg_special/; cp /var/lib/apt/extended_states /root/svg_special/; dpkg --get-selections "*" > /root/svg_special/dpkg_get_selection; cp -R /etc /root/svg_special/etc
  2. Ensuite il est conseillé d’utiliser screen pour pouvoir se reconnecter (avec screen -r) à en cas de déconnexion :
    screen
  3. On va effectuer une mise à jour des paquets avec apt-get update && apt-get upgrade
  4. Le processus de mise à niveau décrit sur le site de Debian a été conçu pour des mises à niveau des systèmes Jessie « purs » sans paquet provenant d’autres sources. Pour une meilleure fiabilité du processus de mise à niveau, vous pouvez supprimer ces paquets du système avant de commencer la mise à niveau. :
    aptitude search '~i(!~ODebian)'
  5. On peut lancer la commande dpkg --audit pour s’assurer que tout est bon avant la migration. On peut également taper dpkg --get-selections "*" | more et vérifier qu’aucun paquet n’est en on hold
  6. Maintenant il faut remplacer tous les « jessie » de /etc/apt/sources.list par des « stretch », ce qui va donner chez moi :
    deb http://ftp.fr.debian.org/debian stretch main non-free
    
    deb http://debian.mirrors.ovh.net/debian/ stretch main
    deb-src http://debian.mirrors.ovh.net/debian/ stretch main
    
    deb http://security.debian.org/ stretch/updates main
    deb-src http://security.debian.org/ stretch/updates main
    

    On vérifiera aussi les autres fichiers qui peuvent se trouver dans /etc/apt/sources.list.d

  7. Si vous utilisez MySQL, sachez qu’avec Stretch vous pourriez passer sur MariaDB… Si vous souhaitez utiliser MySQL, on peut se référer à ce blog post
    • On ajoute/édite le fichier source pour mysql : nano /etc/apt/sources.list.d/mysql.list
    • On y ajoute les sources suivantes (pour mysql-5.7) :
      deb http://repo.mysql.com/apt/debian/ stretch mysql-5.7
      deb-src http://repo.mysql.com/apt/debian/ stretch mysql-5.7
      
    • On ajoute la clé publique de ce repo :
      wget -O /tmp/RPM-GPG-KEY-mysql https://repo.mysql.com/RPM-GPG-KEY-mysql
      apt-key add /tmp/RPM-GPG-KEY-mysql
      rm /tmp/RPM-GPG-KEY-mysql
      
  8. Il est recommandé d’utiliser le programme /usr/bin/script pour enregistrer une transcription de la session de mise à niveau. Ainsi, quand un problème survient, on a un enregistrement de ce qui s’est passé. Pour démarrer un enregistrement, taper :
    script -t 2>~/upgrade-stretch.time -a ~/upgrade-stretch.script
  9. On passe aux choses sérieuses, en commençant par mettre à jour les listes des paquets :
    apt-get update
  10. On va vérifier qu’on a la place suffisante (un message explicite apparait sinon) :
    apt-get -o APT::Get::Trivial-Only=true dist-upgrade
  11. On va maintenant faire une mise à jour minimale :
    apt-get upgrade
  12. Et à partir de là le système va vous questionner… en général choisir l’option par défaut si vous ne savez pas quoi répondre
  13. Puis on continue avec
    apt-get dist-upgrade

Cette dernière étape va durer un certain temps. Une fois terminé, vous pouvez redémarrer le serveur pour s’assurer que tout va bien.

Avec phpmyadmin vous pourriez recevoir l’erreur suivante :

PHP Warning: require_once(): open_basedir restriction in effect. File(/usr/share/php/php-php-gettext/gettext.inc) is not within the allowed path(s): (/usr/share/phpmyadmin/:/etc/phpmyadmin/:/var/lib/phpmyadmin/:/usr/share/php/php-gettext/) in /usr/share/phpmyadmin/libraries/common.inc.php on line 77
PHP Warning: require_once(/usr/share/php/php-php-gettext/gettext.inc): failed to open stream: Operation not permitted in /usr/share/phpmyadmin/libraries/common.inc.php on line 77

Dans ce cas là, il faut rajouter /usr/share/php/php-php-gettext/ dans le fichier /etc/phpmyadmin/apache.conf sur la ligne open_base_dir. Ce qui va donner la ligne : php_admin_value open_basedir /usr/share/phpmyadmin/:/etc/phpmyadmin/:/var/lib/phpmyadmin/:/usr/share/php/php-gettext/:/usr/share/php/php-php-gettext/

J’ai eu quelques soucis avec fail2ban qui ne démarrait pas. J’ai pu résoudre en consultant le Troubleshooting de leur wiki. Il s’agissait du fichier /etc/fail2ban/jail.d/defaults-debian.conf qui avait été créé avec une entrée invalide. Il a suffit de le supprimer.

Une fois les erreurs corrigées, on va nettoyer tous les paquets avec

apt-get autoremove