1. Serveur Debian

Sur le serveur, on installe WireGuard avec apt install wireguard -y.

2. Générer la clé privée et publique du serveur

Pour cela, on va faire :

sudo wg genkey | sudo tee /etc/wireguard/server_private.key
sudo cat /etc/wireguard/server_private.key | sudo wg pubkey | sudo tee /etc/wireguard/server_public.key

On retrouvera nos clés dans les fichiers /etc/wireguard/server_private.key et /etc/wireguard/server_public.key

3. Générer la clé privée et publique du client

Sur le serveur Debian, on va générer des clés pour le client :

wg genkey | tee client_private.key
cat client_private.key | wg pubkey | tee client_public.key

On retrouvera nos clés dans les fichiers ./client_private.key et ./client_public.key

4. Configuration du serveur

On va entrer la configuration suivante dans le fichier /etc/wireguard/wg0.conf :

[Interface]
Address = 10.0.0.1/24
PostUp = iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
PostDown = iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE
ListenPort = 51820
PrivateKey = PRIVATE_KEY_DU_SERVEUR

[Peer]
PublicKey = PUBLIC_KEY_DU_CLIENT
AllowedIPs = 10.0.0.2/32
PersistentKeepalive = 25

5. Activer le transfert IP sur le serveur

Éditer le fichier /etc/sysctl.conf afin d’avoir :

net.ipv4.ip_forward=1
net.ipv6.conf.all.forwarding=1

Et on applique les changements avec la commande sudo sysctl -p

6. Démarrage de WireGuard

Pour démarrer le serveur VPN on tape sudo systemctl start wg-quick@wg0

On peut voir le statut avec la commande sudo systemctl status wg-quick@wg0

7. Configuration du client

Sur notre client (ici un iPad), on installe l’application WireGuard depuis le App Store.

Ensuite, sur notre serveur, on va créer le fichier de configuration client.conf qui sera utilisé par le client, avec le contenu suivant :

[Interface]
PrivateKey = PRIVATE_KEY_DU_CLIENT
Address = 10.0.0.2/32
DNS = 8.8.8.8, 8.8.4.4 # on utilise les DNS de Google

[Peer]
PublicKey = PUBLIC_KEY_DU_SERVEUR
Endpoint = mon_serveur.debian.home:51820 # on indique l'IP/hostname de notre serveur VPN
AllowedIPs = 0.0.0.0/0, ::/0
PersistentKeepalive = 25

Afin de transmettre cette configuration à l’iPad, on peut générer un QR code. Pour cela on installe ce qu’il faut : sudo apt install qrencode, puis on génère avec qrencode -t ansiutf8 < client.conf

Sur l'iPad, on ouvre l'application WireGuard puis on ajoute un client en utilisant le QR Code généré.

8. Vérification

En activant le VPN sur l'iPad, on peut tester si tout fonctionne comme prévu en vérifiant l'adresse IP de l'iPad.
Sur le serveur, on peut utiliser la commande wg show pour voir un peu ce qu'il se passe.

To have a good version of Android TV 9:

1. Downlad a VirtualBox image of it – I used the one shared in this video
2. Install VirtualBox if you don’t have it yet
3. Load the downloaded image with VirtualBox
4. Configure it by going to the network section and select “Bridged Adapter” instead of “NAT”
5. Once Android TV is started, you’ll have to connect to your Google account, but using the keyboard might be challenging – you can right click on the USB cable icon that appears at the bottom right of the VirtualBox window, and select your keyboard (be aware it’s a QWERTY map that is used)

After that, go to the Android TV settings, in the “About” section, and click several times on the Build Version to enable the Developer Mode. From the developer menu, you can enable the “Debug USB”.

Once everything is correctly configured, the Android TV should be visible on your local network, meaning you can use the ADB command to test your app.

Start a new project

1. In a new folder, type npm init to start a new project.
2. Make sure to use a Node >= v14 (I use Node v18 – and Volta can be useful to manage several versions of Node for Windows)
3. Install some dependencies: npm install axios @azure/msal-node uuid

Certificate

In the past, we could use the add-in application feature to get a token, but Microsoft announced it will be retired.

We now need to pass through an Azure application to get it. But before creating the Azure app, we need to create a Private Certificate key.

1. Create the file Create-SelfSignedCertificate.ps1 using the below code (source):

#Requires -RunAsAdministrator
<#
.SYNOPSIS
Creates a Self Signed Certificate for use in server to server authentication
.DESCRIPTION
Source: https://learn.microsoft.com/en-us/sharepoint/dev/solution-guidance/security-apponly-azuread#setting-up-an-azure-ad-app-for-app-only-access
.EXAMPLE
.\Create-SelfSignedCertificate.ps1 -CommonName "MyCert" -StartDate 2015-11-21 -EndDate 2017-11-21
This will create a new self signed certificate with the common name "CN=MyCert". During creation you will be asked to provide a password to protect the private key.
.EXAMPLE
.\Create-SelfSignedCertificate.ps1 -CommonName "MyCert" -StartDate 2015-11-21 -EndDate 2017-11-21 -Password (ConvertTo-SecureString -String "MyPassword" -AsPlainText -Force)
This will create a new self signed certificate with the common name "CN=MyCert". The password as specified in the Password parameter will be used to protect the private key
.EXAMPLE
.\Create-SelfSignedCertificate.ps1 -CommonName "MyCert" -StartDate 2015-11-21 -EndDate 2017-11-21 -Force
This will create a new self signed certificate with the common name "CN=MyCert". During creation you will be asked to provide a password to protect the private key. If there is already a certificate with the common name you specified, it will be removed first.
#>
Param(

[Parameter(Mandatory=$true)]
   [string]$CommonName,

[Parameter(Mandatory=$true)]
   [DateTime]$StartDate,

[Parameter(Mandatory=$true)]
   [DateTime]$EndDate,

[Parameter(Mandatory=$false, HelpMessage="Will overwrite existing certificates")]
   [Switch]$Force,

[Parameter(Mandatory=$false)]
   [SecureString]$Password
)

# DO NOT MODIFY BELOW
function CreateSelfSignedCertificate(){
  #Remove and existing certificates with the same common name from personal and root stores
    #Need to be very wary of this as could break something
    if($CommonName.ToLower().StartsWith("cn="))
    {
        # Remove CN from common name
        $CommonName = $CommonName.Substring(3)
    }
    $certs = Get-ChildItem -Path Cert:\LocalMachine\my | Where-Object{$_.Subject -eq "CN=$CommonName"}
    if($certs -ne $null -and $certs.Length -gt 0)
    {
        if($Force)
        {

foreach($c in $certs)
            {
                remove-item $c.PSPath
            }
        } else {
            Write-Host -ForegroundColor Red "One or more certificates with the same common name (CN=$CommonName) are already located in the local certificate store. Use -Force to remove them";
            return $false
        }
    }

$name = new-object -com "X509Enrollment.CX500DistinguishedName.1"
    $name.Encode("CN=$CommonName", 0)

$key = new-object -com "X509Enrollment.CX509PrivateKey.1"
    $key.ProviderName = "Microsoft RSA SChannel Cryptographic Provider"
    $key.KeySpec = 1
    $key.Length = 2048
    $key.SecurityDescriptor = "D:PAI(A;;0xd01f01ff;;;SY)(A;;0xd01f01ff;;;BA)(A;;0x80120089;;;NS)"
    $key.MachineContext = 1
    $key.ExportPolicy = 1 # This is required to allow the private key to be exported
    $key.Create()

$serverauthoid = new-object -com "X509Enrollment.CObjectId.1"
    $serverauthoid.InitializeFromValue("1.3.6.1.5.5.7.3.1") # Server Authentication
    $ekuoids = new-object -com "X509Enrollment.CObjectIds.1"
    $ekuoids.add($serverauthoid)
    $ekuext = new-object -com "X509Enrollment.CX509ExtensionEnhancedKeyUsage.1"
    $ekuext.InitializeEncode($ekuoids)

$cert = new-object -com "X509Enrollment.CX509CertificateRequestCertificate.1"
    $cert.InitializeFromPrivateKey(2, $key, "")
    $cert.Subject = $name
    $cert.Issuer = $cert.Subject
    $cert.NotBefore = $StartDate
    $cert.NotAfter = $EndDate
    $cert.X509Extensions.Add($ekuext)
    $cert.Encode()

$enrollment = new-object -com "X509Enrollment.CX509Enrollment.1"
    $enrollment.InitializeFromRequest($cert)
    $certdata = $enrollment.CreateRequest(0)
    $enrollment.InstallResponse(2, $certdata, 0, "")
    return $true
}

function ExportPFXFile()
{
    if($CommonName.ToLower().StartsWith("cn="))
    {
        # Remove CN from common name
        $CommonName = $CommonName.Substring(3)
    }
    if($Password -eq $null)
    {
        $Password = Read-Host -Prompt "Enter Password to protect private key" -AsSecureString
    }
    $cert = Get-ChildItem -Path Cert:\LocalMachine\my | where-object{$_.Subject -eq "CN=$CommonName"}

    Export-PfxCertificate -Cert $cert -Password $Password -FilePath "$($CommonName).pfx"
    Export-Certificate -Cert $cert -Type CERT -FilePath "$CommonName.cer"
}

function RemoveCertsFromStore()
{
    # Once the certificates have been been exported we can safely remove them from the store
    if($CommonName.ToLower().StartsWith("cn="))
    {
        # Remove CN from common name
        $CommonName = $CommonName.Substring(3)
    }
    $certs = Get-ChildItem -Path Cert:\LocalMachine\my | Where-Object{$_.Subject -eq "CN=$CommonName"}
    foreach($c in $certs)
    {
        remove-item $c.PSPath
    }
}

if(CreateSelfSignedCertificate)
{
    ExportPFXFile
    RemoveCertsFromStore
}

2. Open a PowerShell console as an administrator
3. Create the certificate with a command like .\Create-SelfSignedCertificate.ps1 -CommonName "SharePointOnlinePrivateKey" -StartDate 2024-04-01 -EndDate 2035-01-01 (if you receive an error, make sure to allow running this kind of script with the command Set-ExecutionPolicy RemoteSigned)
4. A password is required (e.g. "HereIsMyPass1223")
5. Two files are created: SharePointOnlinePrivateKey.pfx and SharePointOnlinePrivateKey.cer

We're going to install OpenSSL to convert the pfx file to a pem:

1. Install OpenSSL (e.g. the light version for Win64) as an administrator
2. Find the OpenSSL installation directory (e.g. C:\Program Files\OpenSSL-Win64\)
3. Open the start.bat file from this OpenSSL directory
4. A command window opens – go to the directory where the SharePointOnlinePrivateKey.pfx file is (e.g. cd C:\Users\Aymeric\Documents\nodejs\spo-experiments)
5. In the OpenSSL command window, type openssl pkcs12 -in SharePointOnlinePrivateKey.pfx -out SharePointOnlinePrivateKey.pem (the password entered in the previous steps will be asked three times)

We should now have a file called SharePointOnlinePrivateKey.pem

Azure Application

It's time to create the related Azure application:

1. Go to https://portal.azure.com
2. Go to the Microsoft Entra ID section
3. Go to the "App Registrations", and click on "New registration"
4. I won't detail all the steps to create the app
5. Give a name to the app (e.g. "SharePoint Online Remote Access")

Get Thumbprint

We need the thumbprint:

1. Go to "Certificates & secrets" section
2. Go to "Certificates" tab (see how) and upload the SharePointOnlinePrivateKey.cer file you created before
3. Once uploaded, it will provide "Thumbprint" (e.g. "F7D8D4F2F140E79B215899BD93A14D0790947789") – copy this value for a later use.

Get Client Id and Tenant Id

We need the clientId and the tenantId:

1. From the overview page of your app, copy the "Application (client) Id" (e.g. "75284292-7515-4e2f-aae9-d1777527dd7b") and the "Directory (tenant) ID" (e.g. "945c177a-83a2-4e80-9f8c-5a91be5752dd")

Platform configuration

Additional configuration required for the application:

1. Go to "Authentication" menu, and under "Platform configurations", click on "Add a Platform"
2. Choose "Web" and enter https://localhost for "Redirect URLs"
3. Choose "Access Token" and "ID Token" in the "Implicit grant and hybrid flows" section

API Permissions

We want to give the permissions to the Azure app to get access to only one (or more) specific Sites Collection, but not all of the tenant site collections. To do so, we only need the Sites.Selected permissions (ref1 and ref2).

First, let's select it from a Microsoft Graph perspective:

1. Go to the "API Permissions" section from the left navigation
2. Click on "Add a permission"
3. Select "Microsoft Graph"
4. Then "Application Permissions"
5. Select "Sites.Selected"

Then, let's select it for SharePoint Online REST perspective:

1. From the same section, select "SharePoint"
2. Then "Application Permissions"
3. Select "Sites.Selected"

Remark: you might need to contact the IT team to get your API Permissions approved/granted.

Get Client Secret

We need the clientSecret:

1. Go to the "Certificates and Secrets" section from the left navigation
2. Go to "Client Secret" and click on "New client secret"
3. In "Description" you can put something like "SharePoint Online Remote Access", and choose 365 days for the expiration
4. Once done, make sure to copy Value (e.g. "rVE7Q~Z1BhRXaljbj7SPg~U2HYJRR-feckrxKbCt") that is our clientSecret

App permissions on Site Collection

It's time to indicate that the Azure application can have access to a specific Site Collection (e.g. https://mycompany.sharepoint.com/sites/contoso).

To proceed, we need the siteId:

1. Go to the SharePoint website (e.g. https://mycompany.sharepoint.com/sites/contoso)
2. Click right in the page to see the source (or CTRL U)
3. Search for the variable siteId (e.g. "7d6d9e18-e8de-4b7c-9582-3f978726f356")

To give the permissions to the app, a special command must be entered (see this video for more information). In theory, the site collection admin can do it, but it might be restricted in your organization, and you'll need the assistance of a tenant admin.

We need the AppId (as known as clientId) and DisplayName (as known as the Azure app name):

$targetSiteUri = 'https://mycompany.sharepoint.com/sites/contoso'
Connect-PnpOnline $targetSiteUri
Grant-PnPAzureADAppSitePermission -AppId '75284292-7515-4e2f-aae9-d1777527dd7b' -DisplayName 'SharePoint Online Remote Access' -Site $targetSiteUri -Permissions Write

Interact with SharePoint

Microsoft Graph only needs a client_secret, while SharePoint REST API needs a client_assertion from the Certificate Private key.

So, let's start with Microsoft Graph to verify our Azure app has the correct access permissions.

Microsoft Graph

Below is the script we can use:

const axios = require('axios');

// our constants that we found previously
// they should not be hard-coded in your script, but store in a safe place
const tenantId = '945c177a-83a2-4e80-9f8c-5a91be5752dd';
const clientId = '75284292-7515-4e2f-aae9-d1777527dd7b';
const clientSecret = 'rVE7Q~Z1BhRXaljbj7SPg~U2HYJRR-feckrxKbCt';
const siteId = '7d6d9e18-e8de-4b7c-9582-3f978726f356';

async function getAccessToken() {
  const resource = 'https://graph.microsoft.com';

  try {
    const response = await axios.post(`https://login.microsoftonline.com/${tenantId}/oauth2/v2.0/token`, new URLSearchParams({
      grant_type: 'client_credentials',
      client_id: clientId,
      client_secret: clientSecret,
      scope: `${resource}/.default`
    }), {
      headers: {
        'Content-Type': 'application/x-www-form-urlencoded'
      }
    });

    return response.data.access_token;
  } catch (error) {
    console.error('Error getting access token:', error);
    throw error;
  }
}

// get a SharePoint item using Graph
getAccessToken()
.then(token => {
  // we need to find the siteId for each level
  // we could use `https://graph.microsoft.com/v1.0/sites/${siteId}/sites` to find the children sites Id
  // mycompany.sharepoint.com/sites/contoso/Toolbox -> 919f3ff8-2cfd-469d-ac2c-cf58475ee72a
  // mycompany.sharepoint.com/sites/contoso/Toolbox/Demo -> 37af7205-ebd1-49e5-a770-cdb182d2ae81
  return axios.get(`https://graph.microsoft.com/v1.0/sites/${siteId}/sites/919f3ff8-2cfd-469d-ac2c-cf58475ee72a/sites/37af7205-ebd1-49e5-a770-cdb182d2ae81/lists/Assets/items/123?expand=fields(select=Title)`, {
    headers:{
      'Authorization':'Bearer '+token,
      'Accept': 'application/json'
    }
  })
})
.then(res => {
  console.log(res.data);
})
.catch(err => {
  console.log("err => ", err.response.data)
})

SharePoint Online REST API

If it worked with the Graph version, it means we can now test with the SharePoint REST API:

const axios = require('axios');
const fs = require("fs");
const crypto = require("crypto");
const msal = require("@azure/msal-node");
const { v4: uuid } = require('uuid');

// our constants that we found previously
// they should not be hard-coded in your script, but store in a safe place
const tenantId = '945c177a-83a2-4e80-9f8c-5a91be5752dd';
const clientId = '75284292-7515-4e2f-aae9-d1777527dd7b';
const clientSecret = 'rVE7Q~Z1BhRXaljbj7SPg~U2HYJRR-feckrxKbCt';
const resource = 'https://mycompany.sharepoint.com';
const privateKeyPassPhrase = "HereIsMyPass1223";
const thumbprint = "F7D8D4F2F140E79B215899BD93A14D0790947789";

// generate the client_assertion
function getClientAssertion () {
  // source: https://github.com/AzureAD/microsoft-authentication-library-for-js/blob/dev/lib/msal-node/docs/certificate-credentials.md
  // decrypt the private key
  const privateKeySource = fs.readFileSync("./SharePointOnlinePrivateKey.pem");
  const privateKeyObject = crypto.createPrivateKey({
      key: privateKeySource,
      passphrase: privateKeyPassPhrase,
      format: "pem",
  });

  const privateKey = privateKeyObject.export({
      format: "pem",
      type: "pkcs8",
  });
  
  const config = {
    auth: {
      clientId: clientId,
      authority: `https://login.microsoftonline.com/${tenantId}`,
      clientCertificate: {
        thumbprint: thumbprint, // a 40-digit hexadecimal string
        privateKey: privateKey,
      },
    },
  };

  // Create msal application object
  const cca = new msal.ConfidentialClientApplication(config);
  const helper = {
    createNewGuid:uuid
  }
  const issuer = clientId;
  const jwtAudience = `https://login.microsoftonline.com/${tenantId}/oauth2/v2.0/token`;
  return cca.clientAssertion.getJwt(helper, issuer, jwtAudience);
}

async function getAccessToken() {
  // see https://github.com/SharePoint/sp-dev-docs/issues/5889 and https://learn.microsoft.com/en-us/entra/identity-platform/v2-oauth2-client-creds-grant-flow#second-case-access-token-request-with-a-certificate
  try {
    const response = await axios.post(`https://login.microsoftonline.com/${tenantId}/oauth2/v2.0/token`, new URLSearchParams({
      client_id: clientId,
      scope: `${resource}/.default`,
      client_assertion_type: "urn:ietf:params:oauth:client-assertion-type:jwt-bearer",
      client_assertion: getClientAssertion(),
      grant_type: 'client_credentials',
    }), {
      headers: {
        'Content-Type': 'application/x-www-form-urlencoded'
      }
    });

    return response.data.access_token;
  } catch (error) {
    console.error('Error getting access token:', error);
    throw error;
  }
}

// run the script
getAccessToken()
.then(token => {
  return axios.get(`https://mycompany.sharepoint.com/sites/contoso/Toolbox/Demo/_api/web/lists/getbytitle('Assets')/items(123)?$select=Title`, {
    headers:{
      'Authorization':'Bearer '+token,
      'Accept': 'application/json; odata=verbose'
    }
  })
})
.then(response => {
  console.log(response.data)
})
.catch(console.log)

• HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System
• HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon

The solution is to use another end point _api/site/getrecyclebinitems (see the documentation for the various parameters):

fetch("https://tenant.sharepoint.com/sites/YourSite/_api/site/getrecyclebinitems?rowLimit=%2770000%27&itemState=0&orderby=3&isAscending=false", {
  "headers": {
    "accept": "application/json",
    "content-type": "application/json",
    "x-requestdigest": document.querySelector("#__REQUESTDIGEST").value
  },
  "method": "GET",
})
.then(res => res.json())
.then(json => {
  console.log(json.value);
  // it will show an array with an object that contains several interesting properties:
  // AuthorEmail, AuthorName, DeletedByEmail, DeletedDate, DirName, Id, LeafName, Title, …
})

You can then filter the result to get what you need, for example if you’re looking for an item from the list “Projects” that has the ID 1981:

json.value.filter(item => item.DirName.includes("Lists/Projects") && item.LeafName === "1981_.000");

Then, to restore an item, we need the Id from the previous result. The famous endpoint to restore an item is _api/site/RecycleBin('Id')/restore, however it could also return the error “The attempted operation is prohibited because it exceeds the list view threshold”. In that case, we can use this other endpoint _api/site/RecycleBin/RestoreByIds:

fetch("https://tenant.sharepoint.com/sites/YourSite/_api/site/RecycleBin/RestoreByIds", {
  "headers": {
    "accept": "application/json",
    "content-type": "application/json",
    "x-requestdigest": document.querySelector("#__REQUESTDIGEST").value
  },
  "method": "POST",
  "body":JSON.stringify({
    "ids": [
      "4f855ee7-472b-414a-a482-4317a114c1a2" // Id to restore
    ],
    "bRenameExistingItems": true
  })
})

async function hmac_sha1 (str, secret) {
  // see https://stackoverflow.com/a/47332317/1134119
  let enc = new TextEncoder("utf-8");
  let key = await window.crypto.subtle.importKey(
    "raw", // raw format of the key - should be Uint8Array
    enc.encode(secret),
    { // algorithm details
      name: "HMAC",
      hash: {name: "SHA-1"}
    },
    false, // export = false
    ["sign", "verify"] // what this key can do
  );
  let signature = await window.crypto.subtle.sign(
    "HMAC",
    key,
    enc.encode(str)
  );
  let b = new Uint8Array(signature);
  return Array.prototype.map.call(b, x => x.toString(16).padStart(2, '0')).join("");
}

We first need a manifest.yml file that contains:

---
applications:
- name: APP-NAME
  buildpack: nodejs_buildpack
  no-route: true
  health-check-type: process
  env:
    OPTIMIZE_MEMORY: true

The no-route parameter is true so that we don’t get a route assigned, and the health-check-type is set to process so that the orchestrator monitors process availability and doesn’t try to ping a non-existent web endpoint. And OPTIMIZE_MEMORY in “env” section is based on the Pivotal recommendations.

If you need to use a local package in your app, you’ll have to pack it up first. To do it, go to your local module folder, and type npm pack. It will create a .tgz file that you’ll have to store in a local_modules folder for your app. Next, use npm install .\local_modules\package-1.2.3.tgz.

You can now deploy your app with pcf push APP-NAME and you can read the logs with cf logs APP-NAME --recent.

There is a workaround with “Transform data using Power Query” (ATTENTION: you cannot load it from a flow from a Solution… you’ll have to go to your Flows and edit the flow from there):

Let’s say we have 3 tables: ITEM_CATALOG, CATALOG and CURRENCY. We want to join them and filter them based on a variable found previously in our flow.

First, we can define our where. Here I have several values that I want to test using a IN:

I create a string with my different values separated by a coma.

Next, we can open the Power Query editor:

In the interface, we choose the 3 tables we need to merge and we add a parameter called “where”:

We rename it to “where” and leave the default settings:

Then we use the “Advance Editor”:

And we wrote the below:

let
  where = Text.Split( "@{variables('where')}" , ",")
in
  where

It means we want to split the variable “where” coming from the flow, based on the coma separator:

We can now merge the tables and add a filter:

And when the step to filter is here, we select “in” and our query:

Last step is to “Enable Load” to make sure this is what the operation will return to our flow:

You can run it to test and see if it works.

Then, to get the output from it, we’ll use a “Parse JSON”… The schema is probably something like:

{
    "type": "object",
    "properties": {
        "resultType": {
            "type": "string"
        },
        "value": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "COLUMN_A": {
                        "type": "string"
                    },
                    "COLUMN_B": {
                        "type": "integer"
                    },
                    "COLUMN_C": {
                        "type": "string"
                    }
                },
                "required": [
                    "COLUMN_A",
                    "COLUMN_B",
                    "COLUMN_C"
                ]
            }
        }
    }
}

You may need to make several tries in order to find the correct schema. You can also use the “Generate from sample” by pasting the data from the previous step:

We use “value” in the loop:

And then we can access our columns:

Important warning: when you use PowerApps to manage your form, all edits to the list settings won’t reflect to the PowerApps form. For example, if you add a field, it won’t show up, and you’ll have to manually update the PowerApps to add it (see at the bottom of this article).

From the list view, go to Integrate then PowerApps and Customize forms:

Once PowerApps has open the form, you need to do several things.

1. Load the ID

We first need to make sure the form will load the required item when we pass the ID URL parameter:

From the SharepointForm Advanced Settings, we change the DefaultMode to check if we have the ID parameter, and if we don’t have it, then it should be a New Form, otherwise an Edit Form:

If(IsBlank(Param("ID")), FormMode.New, FormMode.Edit)

From the SharepointForm Advanced Settings, we change the Item section to check if we have the ID parameter, and if we have it, then we do a lookup in our list to find the data:

If(IsBlank(Param("ID")), SharePointIntegration.Selected, LookUp('NAME OF THE LIST', ID = Value(Param("ID"))))

Add a SUBMIT button

With PowerApps, there is no button to save the changes! We’ll add a button in the form:

In the button’s properties, we change the onSelect to be:

SubmitForm(SharePointForm1)

Be aware that the page will stay with the form after clicking on the button. You could want to close using the Exit() function, but the user will be redirected on office.com … I’d recommend to use Launch() by redirecting your users to a page:

SubmitForm(SharePointForm1); Launch("https://tenant.sharepoint.com/sites/MySite/");

Set field’s value based on URL parameter

We can finally set the field’s value based on the parameter in the URL. Select the INPUT zone of the field, and in the Default section we use the below formula:

If(IsBlank(Param("Title")), ThisItem.Title, Param("Title"))

Here my field is called “Title” so I decided to use a parameter called “Title” as well.

Link to the form

We cannot use the NewForm.aspx or EditForm.aspx to access this form, but we need a special link.

Go to your list settings:

Then go to the form settings (it’s from there that you can decide to keep PowerApps or use the original Sharepoint Forms), and click on See versions and usage:

You’ll get the App Id from this page:

Next, you’ll use the App Id to forge the URL: https://apps.powerapps.com/play/providers/Microsoft.PowerApps/apps/APP_ID
With our example, the URL will be: https://apps.powerapps.com/play/providers/Microsoft.PowerApps/apps/c6f23ac1-dcbd-4daf-925e-2701ab241ca0

You can now pass the URL parameter: https://apps.powerapps.com/play/providers/Microsoft.PowerApps/apps/APP_ID?Title=Hello%20World
And an ID to retrieve an existing item: https://apps.powerapps.com/play/providers/Microsoft.PowerApps/apps/APP_ID?Title=Hello%20World&ID=2

How to use it with a LookUp column?

If you want to auto-select a LookUp field using an URL parameter, you need to do a few things…

First, we need to add the related table. To do so, click on Data in the left navigation bar and search for SharePoint:

Search for the table and add it.

Second (optional) step: click on the Lookup field in the form and change the Items to show a list of options – if no “Lookup” ID in the URL, then we use the default list of options:

The below formula permits to retrieve the “ID” and “Title” from the distant list, base on the “Lookup” parameter, and to rename the result as {Id:"ID", Value:"Title"}:

If(IsBlank(Param("Lookup")), Choices([@'CURRENT LIST NAME'].COLUMN_NAME), RenameColumns(ShowColumns(Filter('DISTANT LIST NAME', ID = Value(Param("Lookup"))), "ID", "Title"), "ID", "Id", "Title", "Value"))

Third, click on the Lookup field in the form and change the DefaultSelectedItems to select the item from the list of options:

The below formula returns an empty selection with {Id:"", Value:""} when no URL param, otherwise it returns the first record for our lookup:

If(IsBlank(Param("Lookup")), {Id:"", Value:""}, First(RenameColumns(ShowColumns(Filter('DISTANT LIST NAME', ID = Value(Param("Lookup"))), "ID", "Title"), "ID", "Id", "Title", "Value")))

And finally, we can pass Lookup=ID in the URL to select the related item in the other list

How to deal with new fields?

If you add a new field to your list’s settings, you’ll have to edit the form in PowerApps, and then edit the fields and add the new one:

(I used this article as a starting point)

But if you prefer to use the ASK CLI, there is some steps to follow…

I first create a fake skill with ask new (using the “hello world” and “AWS Lambda” options).

Once the folder structure and files are created, I edit the .ask/ask-states.json file to reflect the information from the skill I created during the guide.

Then in the folder skill-package I remove everything except skill.json. To find what to put into that file, use the command: ask smapi get-skill-manifest -s <SKILL ID> and copy/paste that code.

Finally, I force the deploy with ask deploy --ignore-hash.

The Lambda function can now be managed locally on your computer and deployed with ASK CLI. You can go to the different skill consoles to delete the fake skill “hello world” you created.

1) Install smalidea plugin

Download the smalidea plugin (see also the related Github Repository).

Open up Android Studio and you should see the welcome screen like the one on screenshot below (if not, close your current project by selecting File -> Close project), go to the Plugins section, and from the wheel icon, select Install Plugin from Disk.... Select the smalidea plugin (ZIP file) you downloaded.

2) Get the third party APK

You first need to know the type of platform where you’ll do your debug tests. To do so, make sure your device is connected to your computer (it could also be a virtual device started from the AVD Manager) with adb devices.
Then, use the command adb shell getprop ro.product.cpu.abi to find the type of processor you have. When I use my phone, I got arm64-v8a.

Go to an APK platform, like https://apkcombo.com/ and search for the Android app you want to debug. Download the APK version that fits to the type you found before:

2bis) Have a look at the APK content

You can use JADX to open the APK and have a quick look at the code.

3) Decompile APK

With APKTool, we’ll use the command: .\apktool.bat d ".\the_original_app_from_apkcombo.com.apk" -o app_to_debug.
A folder called app_to_debug is created with the decompiled version of the application.

Next, we need to copy the source files: create a folder called “src” in the new app_to_debug folder, and type cp -R smali*/* src/.

4) Import project in Android Studio

Open an existing Android Studio project and select the app_to_debug folder where you unpacked APK.

Once the project loads, you need to tell the IDE where is your source code. Make sure you’re using the “Project view” in the left side panel:

Now you can see folder structure in your left panel. Find src/ subfolder right click it and select Mark Directory as -> Sources Root.

5) Prepare App for Debugging

Open AndroidManifest.xml from the app_to_debug and find the XML element <application>. Add the attribute android:debuggable with value “true”. Example:

<application android:debuggable="true" android:allowBackup="true" android:icon="@mipmap/ic_launcher" android:label="@string/app_name" android:largeHeap="true" android:name="org.horaapps.leafpic.App" android:theme="@style/Theme.AppCompat">

6) Repack to APK

You can now repack to APK with the command .\apktool.bat b -d ".\app_to_debug\" -o app_unsigned.apk

7) Sign the APK

7a) Create a keystore

You first need a keystore using keytool and type the below command:
keytool -genkeypair -v -keystore mykey.keystore -alias mykey -keyalg RSA -keysize 2048 -validity 10000

Several questions you’ll be asked, as well as a password. Make sure to remember the password for later.

7b) Validate the APK

You then need zipalign that can be found in the Android SDK folder (e.g. C:\Users\USERNAME\AppData\Local\Android\Sdk\build-tools\31.0.0\zipalign.exe) to validate your APK:
.\Path\to\Android\Sdk\build-tools\31.0.0\zipalign.exe -f -v 4 .\app_unsigned.apk .\app_ready.apk

7c) Sign the APK

Finally you can sign the new created APK with apksigner:
.\Path\to\Android\Sdk\build-tools\31.0.0\apksigner.bat sign --ks .\mykey.keystore --ks-key-alias app_to_debug --out .\app_signed.apk .\app_ready.apk

8) Install the APK

You can install it using adb install app_signed.apk

9) Prepare the host

On your Android device, go to Settings -> Developer options and set USB debugging and Wait for debugger options on. The latter is optional but useful as it allows you wait for debugger connection and not to run app yet.

Finally, you should tap on Select debug app and choose the app you just installed. After all of these, your Developer options menu should look somewhat like this:

Now, launch the app on the Android device, and you’ll get the below message:

10) Forward debugger port

You can use the adb’s port forwarding feature and forward JDWP service where application’s debug interface is listening.

Find the JDWP port with the command adb jdwp, then use this port with the command:
adb forward tcp:5005 jdwp:JDWP_PORT

11) Connect Debugger

Go to Android Studio and from its top menu bar choose Run -> Debug…, then a small message appears with one unique option that is Edit Configurations.... There, in the window, use a plus (+) button at the opt left, and add a new configuration of type Remote. Leave the default configuration as is. Click the Debug button and your app should be running with the attached debugger which means it will stop once a breakpoint is hit and you can investigate the content of app’s variables.

See this wiki page for details: https://github.com/Aymkdn/assistant-freebox-cloud/wiki/Capacitor-Plugin-for-HTTP-requests-with-self-signed-SSL-certificates

Now I want to remove just one domain from this certificate, and it becomes complicated to understand how to do it. The best solution is to create a new certificate for each of my domains, and then to delete the original certname.

Let’s say my certname is called www.example.com and it contains the below domains:

• www.example.com
• example.com
• blog.example.com
• other-example.com
• www.other-example.com
• my-other-domain.com
• www.my-other-domain.com
• api.test.com

The one I don’t need anymore is *.my-other-domain.com.

First, we create a certificate individually for each domain that we want to keep:

certbot --apache --cert-name example.com -d example.com,www.example.com,blog.example.com
certbot --apache --cert-name other-example.com -d other-example.com,www.other-example.com
certbot --apache --cert-name test.com -d api.test.com

--cert-name permits to give our own name to the certificate, and -d indicates which domains should be added to this certificate.

Then we can list all our certificates:

certbot certificates

Using the above command you can find the Certificate Path and now we can delete our original certificate:

certbot revoke --cert-path /etc/letsencrypt/live/www.example.com/fullchain.pem

You’re all set! All your domains should still have a correct certificate, and you revoked the ones you don’t need anymore.

A few steps:

sudo apt-get install software-properties-common dirmngr
wget -qO - https://mariadb.org/mariadb_release_signing_key.asc | sudo apt-key add -
nano /etc/apt/sources.list.d/mariadb.list

In mariadb.list we add the two below lines:

deb [arch=amd64,i386,ppc64el] http://mirror.23media.de/mariadb/repo/10.4/debian stretch main
deb-src http://mirror.23media.de/mariadb/repo/10.4/debian stretch main

Then:

apt-get update
apt-get install mariadb-server

Hopefully some of the operations can be done with REST API, and in my organization the threshold limitation is removed during the night (1am to 4am). By combining a schedule service (like node-schedule) with my library SharepointPlus I can run this admin tasks during the night while my virtual computer is running.

To delete a column

$SP().ajax({
  url:"https://company.com/sharepoint/team/sales/_api/web/lists/getbytitle('My List')/fields/getbytitle('Column To Delete')",
  method: "POST",
  headers: {
    "IF-MATCH": "*",
    "X-HTTP-Method": "DELETE"
  }
});

To index a column

$SP().ajax({
  url:"https://company.com/sharepoint/team/sales/_api/web/lists/getbytitle('My List')/fields/getbytitle('Column to Index')",
  method: "POST",
  body:JSON.stringify({
    "__metadata":{ type: "SP.Field" },
    "Indexed":true
  }),
  headers: {
    "IF-MATCH": "*",
    "X-HTTP-Method": "MERGE"
  }
})

To delete a list

$SP().ajax({
  url:"https://company.com/sharepoint/team/sales/_api/web/lists/getbytitle('List To Delete')",
  method: "POST",
  headers: {
    "IF-MATCH": "*",
    "X-HTTP-Method": "DELETE"
  }
})

To delete a list

$SP().ajax({
  url:"https://company.com/sharepoint/team/sales/_api/web/lists/getbytitle('List To Delete')",
  method: "POST",
  headers: {
    "IF-MATCH": "*",
    "X-HTTP-Method": "DELETE"
  }
})

To delete a site

$SP().ajax({
  url:"https://company.com/sharepoint/team/sales/website_to_delete/_api/web/",
  method: "POST",
  headers: {
    "X-HTTP-Method": "DELETE"
  }
})

• https://stackoverflow.com/questions/44988163/create-self-signed-certificate-for-testing-localhost-and-have-it-accepted-by-the
• http://woshub.com/how-to-create-self-signed-certificate-with-powershell/
• https://stackoverflow.com/questions/4691699/how-to-convert-crt-to-pem/4691749#4691749
• https://webpack.js.org/configuration/dev-server/#devserver-https

If you develop with Webpack under Windows and you want to open your localhost server in HTTPS with IE11 you may receive a message like :

“Content was blocked because it was not signed by a valid security certificate.”

Below I explain the steps to make it work with IE11:

1. Open Powershell in Administrator mode
2. Type (no need to change anything, just copy/paste):

   New-SelfSignedCertificate -certstorelocation cert:\localmachine\my -dnsname localhost -FriendlyName "Dev localhost" -NotAfter (Get-Date).AddMonths(240) -TextExtension @("2.5.29.37={text}1.3.6.1.5.5.7.3.1")

   It should return a Thumbprint:

      PSParentPath: Microsoft.PowerShell.Security\Certificate::LocalMachine\my
   
   Thumbprint                                Subject
   ----------                                -------
   1D97DFF290FBACE0871F16AD1B38172F253CA048  CN=localhost

   You may want to open certlm.msc to see the new certificate in Personal

3. Export to .cer with the below Powershell command (make sure to replace 1D97DFF290FBACE0871F16AD1B38172F253CA048 with the Thumbprint you received before)… it will be exported to C:\localhost.cert (but you can change the path):

   Export-Certificate -Cert Cert:\LocalMachine\My\1D97DFF290FBACE0871F16AD1B38172F253CA048 -FilePath C:\localhost.cer (example: Export-Certificate -Cert Cert:\LocalMachine\My\PREVIOUS_Thumbprint_HERE -FilePath C:\localhost.cer)

4. We now need to export to .pfx with the below Powershell commands:

   $CertPassword = ConvertTo-SecureString -String "YourPassword" -Force –AsPlainText
   Export-PfxCertificate -Cert cert:\LocalMachine\My\2779C7928D055B21AAA0Cfe2F6BE1A5C2CA83B30 -FilePath C:\localhost.pfx -Password $CertPassword
   

5. Next we’ll use the opensource application called XCA: download and install it.
6. Open XCA and go to : File > New Database
7. Choose where you want to have this database and provide a password (to make it easy, you can choose the same “YourPassword” than before)
8. Go to tab Certificates
9. Click Import and select the previous created localhost.cer
10. Once imported, right click the cert and Export > File, then select “*.crt” for the file format
11. Go to the Private Keys tab and click on Import PFX
12. Select the previously created PFX and click Import All
13. Right click on “localhost_key” and Export > File, then select “PEM Private (*.pem)” for the file format
14. Close XCA and go to the folder with all your files
15. Double-click on the file localhost.crt you have just created
16. Windows shows a popup with a "Install Certificate..." button; Click on this button
17. Choose Current User, and next, click on “Place all certificates in the following store”, and browse to “Trusted Root Certification Authorities”
18. Finally, in your Webpack config, you should enter something like the below:

    module.exports = {
      //...
      devServer: {
        https: {
          key: fs.readFileSync('C:\\localhost.pem'),
          cert: fs.readFileSync('C:\\localhost.crt')
        }
      }
    };
    

Launch your webpack server and open IE to your localhost server. It should work…

If you need to do CORS request, you may need to open IE settings, then go to Security and make sure to add localhost in the Local Intranet/Trusted Sites, then click on “Custom Level…”, go to “Miscellaneous” and select “Enable” for “Access data sources across domains”.

When the users download an Office document from this library and then re-upload it we could have the below error message:

There is at least one lookup column that enforces a relationship behavior and contains values that reference one or more non-existent items in the target list.

It’s because the Office documents keep the custom columns from the document library from where they have been downloaded… In that case the file tries to reassign the Metadata with an ID that doesn’t exist anymore… causing this issue!

Because I’m using a homemade interface to upload documents, I’ve been able to pass some code to delete the file’s properties on-the-fly before pushing it into the document library.

To do so, you need JSZip that will permit to unzip the Office document in order to retrieve the file docProps/custom.xml and to change the properties we want before the final upload.

Let’s imagine my page contains an <input type="file">, and that I have already loaded JSZip. Then I use FileReader:

var fileReader = new FileReader();
fileReader.onloadend = function(e) {
  // content is the "arraybuffer" that represents my file
  var content = e.target.result;
  // check the file's extension (here "docx", "xlsx", and "pptx", but we could add more extensions
  var ext=file.name.split(".").slice(-1)[0];
  switch (ext) {
    case "docx":
    case "xlsx":
    case "pptx": {
      // load content in JSZip
      JSZip.loadAsync(content)
      .then(function(zip) {
        // unzip the file that contains metadata/properties
        return zip.file("docProps/custom.xml").async("text")
        .then(function(txt) {
          // replace the faulty column
          txt = txt.replace(/name="Metadata"><vt:lpwstr>\d+<\/vt:lpwstr>/,'name="Metadata"><vt:lpwstr><\/vt:lpwstr>');
          // reinject the new file
          zip.file("docProps/custom.xml", txt);
          return zip.generateAsync({type:"arraybuffer"})
        })
      })
      .then(function(content) {
        // do something with your content
        // for example (https://aymkdn.github.io/SharepointPlus/): $SP().list("my_list").createFile({content:content, filename:file.name})
      })
      break;
    }
    default: {
      // for the other files, treat them normally
      // for example (https://aymkdn.github.io/SharepointPlus/): $SP().list("my_list").createFile({content:content, filename:file.name})
    }
  }
}
fileReader.onerror = function(e) {
  console.error(e.target.error);
}
fileReader.readAsArrayBuffer(file);

This protection can be useful, but also very annoying, for example when the file is not closed properly, then the lock could stay “forever”.

There are many posts on the web about it.

I found one that has been super useful: https://pholpar.wordpress.com/2014/04/07/how-to-use-javascript-to-delete-short-term-locks-from-documents-opened-from-sharepoint/
The author explains very well the different steps what I’m trying to summarize:

1. Send a request to _vti_bin/_vti_aut/author.dll with special headers/body
2. Auth.dll will provide the lockid
3. Send a request to _vti_bin/cellstorage.svc/CellStorageService with special headers/body, included the lockid
4. The file is unlocked

The code to send to CellStorageService, and provided by the author, didn’t work for me. I’ve had to use Fiddler and open the document into Office on my computer to see the kind of requests send by it to unlock a document. Based on it, I’ve re-built the code and you can find my solution below.

Tested on Sharepoint 2013 On-Promise only. I don’t know if this solution works for Sharepoint Online or other version.
Also note that I use $SP().ajax() from SharepointPlus, but it’s equivalent to the $.ajax from jQuery.

// full path to the document
var docUrl = "https://website.com/website/Doc_Library/Test.docx";

// start by querying author.dll to find the lockid and the user who locked it
$SP().ajax({
  url: 'https://website.com/website/_vti_bin/_vti_aut/author.dll',
  headers:{
    "Content-Type": "application/x-www-form-urlencoded",
    "MIME-Version": "1.0",
    "Accept": "auth/sicily",
    "X-Vermeer-Content-Type": "application/x-www-form-urlencoded"
  },
  body: 'method=getDocsMetaInfo%3a14%2e0%2e0%2e6009&url%5flist=%5b' + encodeURIComponent(docUrl) + '%5d&listHiddenDocs=false&listLinkInfo=false',
}).then(function(source) {
  // go thru the source page returned to find the lockid and current user
  var nextLine = false;
  var ret = { "lockid":"", "user":"", when:"" };
  source.split("\n").forEach(function(line) {
    if (line.indexOf("vti_sourcecontrollockid") !== -1) nextLine="lockid"; // vti_sourcecontrollockid -> the lockid to use later
    else if (line.indexOf("vti_sourcecontrolcheckedoutby") !== -1) nextLine="user"; // vti_sourcecontrolcheckedoutby -> username of the user who locked it
    else if (line.indexOf("vti_sourcecontrollockexpires") !== -1) nextLine="when"; // vti_sourcecontrollockexpires -> when the server is supposed to unlock it
    else if (nextLine !== false) {
      ret[nextLine] = line.slice(7).replace(/&#([0-9]|[1-9][0-9]|[[01][0-9][0-9]|2[0-4][0-9]|25[0-5]);/g, function (str, match) { return  String.fromCharCode(match); });
      nextLine = false;
    }
  });

  if (!ret.lockid) { alert("Not Locked") }
  else {
    // compose a request based on what Microsoft Office sends to the Sharepoint server
    // found using Fiddler
    var releaseLockReq = '<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"><s:Body><RequestVersion Version="2" MinorVersion="2" xmlns="http://schemas.microsoft.com/sharepoint/soap/"/><RequestCollection CorrelationId="{96A244BD-D13B-4696-9355-231FB673BC4F}" xmlns="http://schemas.microsoft.com/sharepoint/soap/"><Request Url="'+docUrl+'" UseResourceID="true" UserAgent="{1984108C-4B93-4EEB-B320-919432D6E593}" UserAgentClient="msword" UserAgentPlatform="win" Build="16.0.8201.2102" MetaData="1031" RequestToken="1"><SubRequest Type="ExclusiveLock" SubRequestToken="1"><SubRequestData ExclusiveLockRequestType="ReleaseLock" ExclusiveLockID="'+ret.lockid+'"/></SubRequest></Request></RequestCollection></s:Body></s:Envelope>';

    // we send it to the webservice cellstorage.svc
    $SP().ajax({
      url:'https://website.com/website/_vti_bin/cellstorage.svc/CellStorageService',
      body:releaseLockReq,
      headers:{
        'Content-Type':'text/xml; charset=UTF-8',
        'SOAPAction': "http://schemas.microsoft.com/sharepoint/soap/ICellStorages/ExecuteCellStorageRequest"
      }
    })
    .then(function(res) {
      if (res.indexOf('ErrorCode="Success"') !== -1) alert("Success") // the file has been unlocked
      else alert("Failed")
    })
  }
})

I hope it will be useful to someone else!

You first need to Create a new view named Unpublished Content. You can sort by Name, and scroll down to the Folders settings and choose Show all items without folders.

We now have all our files in the view.

Next, switch to the Library ribbon and choose Modify in SharePoint Designer (Advanced) from the Modify View dropdown.

Sharepoint Designer will open your page.

In Sharepoint Designer, switch to the Home tab and choose Advanced Mode. The file is reloaded.

Search into your code the tag ListViewXml. On this line you should have <Query;&>. Just after it, we need to add:
<Where><Or><Eq><FieldRef Name="_ModerationStatus" /><Value Type="ModStat">Draft</Value></Eq><IsNotNull><FieldRef Name="CheckoutUser" /></IsNotNull></Or></Where>

Before:

After:
(click to enlarge)

Save, and your view should only show the unpublished content!

(Article inspired by https://thechriskent.com/2013/02/14/unpublished-view/)

I’m going to summarize the steps from the above article:

1. Download and Install Fiddler on your computer
2. Once Fiddler is installed, launch it and:
   • Click menu Tools > Options, then select the Connections tab
   • Make note of the “Fiddler listens on” port (normally it’s 8888)
   • Make sure the check box for “Allow remote computer to connect” is checked
   • Switch to the HTTPS tab
   • Make sure the check boxes for “Capture HTTPS Connects” and “Decrypt HTTPS traffic” are both checked
   • Restart Fiddler
3. Go to your Android phone then:
   • Tap on Settings, then Wi-Fi
   • Find the network on which you’re connected (normally the first one listed), then tap and hold
   • Choose Modify network from the pop-up
   • Scroll down and enable “Show advanced options”
   • Change “Proxy settings” to Manual
   • Under “Proxy host name” enter the Windows PC IP address from above
   • Under “Proxy port” enter the Fiddler port from above (usually 8888)
   • Tap Save and wait a moment for the network to reconnect
4. Now we need to add the certificate in Android to have the HTTPS working:
   • On Android start the Chrome browser
   • Navigate to http://IP_ADDRESS_WHERE_FIDDLER_IS:8888/ or http://ipv4.fiddler:8888
   • Tap on the link for the “Fiddler Root Certificate”
   • Name the certificate “Fiddler” and install it (entering your PIN or password if prompted)

You’re now ready to capture the traffic on Fiddler!

Once you’re done you can switch back to normal by following the below steps:

1. Tap on Settings, then Wi-Fi
2. Find the network on which you’re connected (should be the first one listed), then tap and hold
3. Choose Modify network from the pop-up
4. Scroll down and select (enable) “Show advanced options”
5. Change “Proxy settings” to None
6. Tap Save and wait a moment for the network to reconnect
7. Go up a level in settings to Security
8. Tap Trusted credentials, then select the User tab
9. Tap on the Fiddler “Do not trust” certificate, then scroll down to remove it
10. You may need to power cycle your device to get all apps to forget about the Fiddler certificate (e.g., the Chrome browser will continue to try to use it for a while)

Je vais essayer d’expliquer brièvement les étapes à suivre pour cela.

1. On va d’abord sauvegarder les données :

   mkdir /root/svg_special; cp -R /var/lib/dpkg /root/svg_special/; cp /var/lib/apt/extended_states /root/svg_special/; dpkg --get-selections "*" > /root/svg_special/dpkg_get_selection; cp -R /etc /root/svg_special/etc

2. Ensuite il est conseillé d’utiliser screen pour pouvoir se reconnecter (avec screen -r) à en cas de déconnexion :

   screen

3. On peut lancer la commande dpkg --audit pour s’assurer que tout est bon avant la migration. On peut également taper dpkg --get-selections "*" | more et vérifier qu’aucun paquet n’est en on hold
4. Maintenant il faut remplacer tous les “wheezy” de /etc/apt/sources.list par des “jessie”, ce qui va donner chez moi :

   deb http://ftp.fr.debian.org/debian jessie main non-free
   
   deb http://debian.mirrors.ovh.net/debian/ jessie main
   deb-src http://debian.mirrors.ovh.net/debian/ jessie main
   
   deb http://security.debian.org/ jessie/updates main
   deb-src http://security.debian.org/ jessie/updates main
   

5. Il est recommandé d’utiliser le programme /usr/bin/script pour enregistrer une transcription de la session de mise à niveau. Ainsi, quand un problème survient, on a un enregistrement de ce qui s’est passé. Pour démarrer un enregistrement, taper :

   script -t 2>~/upgrade-jessie.time -a ~/upgrade-jessie.script

6. On passe aux choses sérieuses, en commençant par mettre à jour les listes des paquets :

   apt-get update

7. On va vérifier qu’on a la place suffisante (un message explicite apparait sinon) :

   apt-get -o APT::Get::Trivial-Only=true dist-upgrade

8. On va maintenant faire une mise à jour minimale :

   apt-get upgrade

9. Et à partir de là le système va vous questionner… en général choisir l’option par défaut si vous ne savez pas quoi répondre
10. Puis on continue avec

    apt-get dist-upgrade

Cette dernière étape va durer un certain temps. Une fois terminé, vous pouvez redémarrer le serveur pour s’assurer que tout va bien.

Après tout ça j’ai rencontré un problème avec la version 2.4 d’Apache, en ayant l’erreur :

AH01630: client denied by server configuration

En cherchant j’ai trouvé des modifications au niveau de la configuration, à savoir qu’il faut mettre Require all granted dans tous les <Directory> des virtual hosts :

<Directory />
  Options FollowSymLinks
  Require all granted
</Directory>

De même, concernant phpmyadmin, il faut modifier /etc/phpmyadmin/apache.conf (avec Require all denied par exemple).

Avec phpmyadmin vous pourriez recevoir l’erreur suivante :

PHP Fatal error: require_once(): Failed opening required ‘./libraries/php-gettext/gettext.inc’ (include_path=’.’) in /usr/share/phpmyadmin/libraries/select_lang.lib.php

Dans ce cas là, il faut rajouter /usr/share/php/php-gettext/ dans le fichier /etc/phpmyadmin/apache.conf sur la ligne open_base_dir (voir superuser.com). Ce qui va donner la ligne : php_admin_value open_basedir /usr/share/phpmyadmin/:/etc/phpmyadmin/:/var/lib/phpmyadmin/:/usr/share/php/php-gettext/

Mon fichier apache2.conf avait été aussi modifié et certains fichiers de configuration n’étaient plus lus (ceux dans le répertoire /etc/apache2/conf.d/). De même il faudra renommer les fichiers présents dans /etc/apache2/sites-available/ en y ajoutant l’extension .conf (et relancer la commande a2ensite sur vos fichiers renommés).

Si vous utilisez des fichiers .htaccess (par exemple avec WordPress), alors assurez vous d’utiliser la balise AllowOverride All … par exemple dans un de vos fichiers de configuration d’Apache il faudra mettre (pour indiquer que tous les .htaccess dans /home/websites/www sont autorisés) :

<Directory /home/websites/www>
   AllowOverride All
</Directory>

Sur un des serveurs il y avait un problème d'encodage avec les fichiers en PHP alors que les HTML n'avaient pas de soucis. Après avoir cherché j'ai découvert que dans le cas de ce serveur je devais modifier /etc/php5/apache2/php.ini pour y mettre default_charset = Off.

Et finalement on va nettoyer tous les paquets avec

apt-get autoremove

Toutes les opérations vont se passer dans une console sur le serveur Linux en mode root. J’utilise ici une Debian 7.10 « Wheezy ».

Je me suis aidé de cet article en anglais.

1. Commencer par nettoyer le contenu de /etc/apache2/sites-available/ en supprimant tous les fichiers inutiles
2. Aller dans /root
3. Installer git (s’il n’est pas déjà installer) : $ apt-get install git
4. Ensuite on tape les commandes suivantes :
   • $ git clone https://github.com/letsencrypt/letsencrypt
   • $ cd letsencrypt
   • $ ./letsencrypt-auto --help
5. Plusieurs programmes vont s’installer.
6. Maintenant on tape : $ ./letsencrypt-auto --apache
7. Si vous obtenez l’erreur Apache plugin support requires libaugeas0 and augeas-lenses version 1.2.0 or higher, please make sure you have you have those installed. alors il va falloir utiliser un backport debian repositories :
   • On commence par éditer le fichier /etc/apt/sources.list en y ajoutant la ligne deb http://ftp.debian.org/debian wheezy-backports main
   • Puis on tape $ apt-get update
   • Et ensuite $ apt-get install -t wheezy-backports libaugeas0 augeas-lenses
   • On peut maintenant refaire $ ./letsencrypt-auto --verbose --apache
8. Une boite de dialogue s’ouvre vous indiquant les domaines trouvés sur votre machine. Par défaut ils sont tous cochés. Suivez les instructions
9. Une fois fait, si vous avez l’erreur urn:acme:error:connection :: The server could not connect to the client to verify the domain :: Failed to connect to host for DVSNI challenge il peut y avoir plusieurs raisons… Pour ma part c’était le firewall qui bloquait le port 443
10. Ou si vous avez une erreur liée à un domaine, assurez-vous que celui-ci pointe bien vers votre box en utilisant la commande nslookup -debug blog.kodono.info 8.8.8.8

Vous devriez maintenant avoir accès à votre site web avec https.

Attention car le certificat Let’s Encrypt expire après 90 jours. Il va donc falloir mettre en place un cron job. On peut prendre celui de https://thealphanerd.io/blog/securing-apache-and-znc-with-letsencrypt/ :

#!/bin/sh
if ! /path/to/letsencrypt/letsencrypt-auto renew > /var/log/letsencrypt/renew.log 2>&1 ; then
  echo Automated renewal failed:
  cat /var/log/letsencrypt/renew.log
  exit 1
fi

On va le placer dans /etc/cron.daily/update-certs (avec les droits 755), et le cron va s’en occuper tout seul.

Si vous continuez à autoriser le HTTP et le HTTPS, alors, pour éviter le mixed content (c’est-à-dire du contenu http qui est appelé sur votre site https) on peut se la jouer brut-force en utilisant un module d’Apache qui va modifier le contenu des pages avant de les renvoyer. À mon sens cela devrait être temporaire, le temps de modifier tous vos fichiers.

1. On va commencer par installer le module substitude: a2enmod substitute
2. Ensuite on utilise le nouveau module pour remplacer les http://notresite en https://notresite, ainsi que les liens extérieurs de http:// en //. Pour cela on modifie nos fichiers .conf dans /etc/apache2/sites-available/ en y ajoutant :

 <Location />
 <If "%{SERVER_PORT} == 443">
 AddOutputFilterByType INFLATE;SUBSTITUTE;DEFLATE text/html text/plain text/xml
 Substitute "s|action=\"http:|action=\"|"
 Substitute "s|action='http:|action='|"
 Substitute "s|src=\"http:|src=\"|"
 Substitute "s|src='http:|src='|"
 Substitute "s|href=\"http:|href=\"|"
 Substitute "s|href='http:|href='|"
 <If>
</Location>

J’utilise <If "%{SERVER_PORT} == 443"> pour n’enclencher la substitution que lorsqu’on demande de l’HTTPS.
Et on oubliera pas de faire un apache2ctl configtest pour vérifier que tout est bon, puis un service apache2 reload pour prendre en compte les modifications !

On crée d’abord le fichier /etc/apt/sources.list.d/mysql.list avec les deux lignes ci-dessous :
deb http://repo.mysql.com/apt/debian/ wheezy mysql-5.6
deb-src http://repo.mysql.com/apt/debian/ wheezy mysql-5.6

2) Clé publique du repo

On crée un fichier mysql.key dans lequel on copie/colle la clé publique de mysql.

Puis on l’ajoute à apt :
apt-key add mysql.key

3) Export / Backup

On va faire une sauvegarde de la base de données et de la configuration de mysql :
mysqldump -u root -pPASSWORD --add-drop-table --routines --events --add-drop-table --all-databases --force > data-for-upgrade.sql
tar cvfvz /root/mysql_conf.tgz /etc/mysql

4) Arrêt du serveur

On stoppe mysql :
mysqladmin -u root -pPASSWORD shutdown

5) On met à jour apt

apt-get update

6) On installe

On installe la nouvelle version :
apt-get install mysql-server-5.6

7) On vérifie

Une fois fait, on lance :
mysql_upgrade -v -u root -pPASSWORD

8) Dernier upgrade

On lance l’upgrade pour que tout soit à jour :
apt-get upgrade

9) Mise à jour de PHP-Mysql

Si vous avez l’erreur mysql_connect(): Headers and client library minor version mismatch, il est recommandé d’installer php5-mysqlnd :
apt-get install php5-mysqlnd

Pour y remédier il faut d’abord définir deux adaptateurs dans l’onglet Network de la VM :

1. NAT (s’assurer que “Cable Connected” est coché)
2. Bridged Adapter (avec “Promiscous Mode”: Allow All, et “Cable Connected” de coché)

Ensuite, au niveau de la machine virtuelle, il devrait y avoir deux connexions :

1. Une qui doit avoir une IP en 10.x.x.x (réseau du VPN)
2. La deuxième qui doit avoir une IP de notre réseau local (192.168.0.x chez moi) … IP Local à définir manuellement si nécessaire, et à noter qu’il faut aussi définir les DNS manuellement (j’utilise ceux de Google) si on veut que ça fonctionne bien

A partir d’ici votre système devrait être en mesure d’accéder au réseau VPN.

(ci-dessous une configuration que je dois faire pour réussir à avoir un semblant d’accès au reste du Net … à noter que ça rend les choses instables)

Dans l’exemple qui suit on va supposer que ma gateway pour le VPN est 10.0.2.2 et pour le réseau local 192.168.0.254

Ensuite j’ai modifié les routes en ouvrant une console cmd en tant qu’administrateur puis je tape :
route print (pour trouver le numéro des interfaces, on va dire que #13 est pour 10.x.x.x et #15 pour l’interface en 192.168.0.x)
route -f (on flush les règles existantes)
route add 0.0.0.0 mask 0.0.0.0 192.168.0.254 IF 15 (par défaut tout le trafic passe par le réseau normal)
route add 10.0.0.0 mask 255.0.0.0 10.0.2.2 IF 13 (et tout ce qui concerne le réseau du VPN on l’envoie vers celui-ci)

Also I tried SDFix and I got the error “Update failed, platform.xml file could not be updated”.

When I tried “ES Explorer”, with Root Mode, I had SuperSU asking for root permission, but then “ES Explorer” said my system is not rooted.

After hours of digging I finally found I had to use that file discovered in that XDA thread. Just download it, unzip, then launch the install.bat ]]> https://blog.kodono.info/wordpress/2015/07/26/android-5-1-1-system-rw-operation-not-permitted/feed/ 0 https://blog.kodono.info/wordpress/2015/05/18/rootdirectory-0x8004100e-wbem_e_invalid_namespace-namespace-specified-cannot-be-found/ https://blog.kodono.info/wordpress/2015/05/18/rootdirectory-0x8004100e-wbem_e_invalid_namespace-namespace-specified-cannot-be-found/#respond Mon, 18 May 2015 19:09:23 +0000 http://blog.kodono.info/wordpress/?p=1508 mofcomp DscCore.mof]]> I think I found the MOF file related to the above error. To fix it you might try:

C:\Windows\System32\wbem>mofcomp DscCore.mof

Finally it was an issue with a MOF file in WMI. To fix it:

C:\Windows\System32\wbem>mofcomp NetAdapterCim.mof

J’ai finalement trouvé comment faire grâce à https://katyscode.wordpress.com/2007/02/03/tutorial-how-to-fix-wmi-corruption/ — il suffit de faire :

C:\Windows\System32\wbem>mofcomp cimwin32.mof
Compilateur MOF Microsoft (R) - Version 6.3.9600.16384
Copyright (c) Microsoft Corp. 1997-2006. Tous droits réservés.
Analyse du fichier MOF : cimwin32.mof
Analyse du fichier MOF effectuée
Stockage des données...
Terminé !

Et un autre lien utile : http://blogs.technet.com/b/askperf/archive/2009/04/13/wmi-rebuilding-the-wmi-repository.aspx

Pour réussir ce que je décris ci-dessous, il vous faudra connaitre votre clé privée et publique de Google Captcha.

Il faut commencer par faire son formulaire (de contact dans mon cas) avec tous les champs voulus via GuiForm, puis on ajoute un champ de type “Heading” avec comme contenu “captcha”. On sauvegarde le formulaire.

Maintenant dans le fichier functions.php de votre thème il faut rajouter :

// on veut rajouter un captcha dans le formulaire de contact
// pour ça on surcharge 'guiform_render_form' qui est appelé par le plugin
function addCaptchaToGuiForm($content) {
  // on va remplacer "<h1>captcha</h1>" par ce qu'il faut
  $content = str_replace("<h1>captcha</h1>", "<div class='g-recaptcha' style='float:right' data-sitekey='VOTRE_CLE_PUBLIQUE' data-theme='light'></div><script src='https://www.google.com/recaptcha/api.js'></script>",$content);
  // on rajoute aussi du JavaScript
  $content .= '<script>'."\r\n".
              "function waitForjQuery() {"."\r\n".
              "  if (typeof jQuery === 'undefined') { setTimeout(waitForjQuery, 50); return }"."\r\n".
              "  var submit = jQuery('.f_submit');"."\r\n".
              "  var html = '<div class=\"'+submit[0].className+'\">'+submit.html()+'</div>';"."\r\n".
              "  submit.before(html).find('button').hide().prop('disabled', true);"."\r\n".
              "  var cloneSubmit = jQuery('.f_submit:first');"."\r\n".
              "  cloneSubmit.on('click', function(event) {"."\r\n".
              "    event.preventDefault();"."\r\n".
              "    jQuery.ajax({"."\r\n".
              "      type: 'GET',"."\r\n".
              "      url: '/wp-content/themes/VOTRE_THEME/checkCaptcha?response='+grecaptcha.getResponse(),"."\r\n".
              "      success: function(data) {"."\r\n".
              "        if (data['success'] === true) {"."\r\n".
              "          jQuery('.f_submit:first').remove()"."\r\n".
              "          jQuery('.f_submit').find('button').prop('disabled',false).show().last().trigger('click');"."\r\n".
              "        }"."\r\n".
              "        else if (data['error-codes']) {"."\r\n".
              "          switch (data['error-codes'][0]) {"."\r\n".
              "            case 'missing-input-secret': "."\r\n".
              "            case 'invalid-input-secret': alert('Erreur : impossible de vérifier le système anti-spam.'); break;"."\r\n".
              "            case 'missing-input-response': "."\r\n".
              "            case 'invalid-input-response': "."\r\n".
              "            default: alert('Erreur : vous devez répondre à l\'anti-spam.');"."\r\n".
              "          }"."\r\n".
              "        }"."\r\n".
              "      },"."\r\n".
              "      dataType: 'json'"."\r\n".
              "    });"."\r\n".
              "  })"."\r\n".
              "}"."\r\n".
              "waitForjQuery();"."\r\n".
              "</script>";
  return $content;
}
add_action('guiform_render_form', 'addCaptchaToGuiForm');

Grâce à cette astuce on modifie notre <h1>captcha</h1> par le code de Google. On ajoute un soupçon de JavaScript qui va permettre de cacher le vrai bouton “Submit”. Lorsqu’on va cliquer sur le faux bouton, le catpcha va être vérifié. Une fois fait le formulaire classique peut-être utilisé.

Attention : dans le code ci-dessus il faut remplacer VOTRE_CLE_PUBLIQUE et VOTRE_THEME par les valeurs correspondantes.

Enfin on a besoin de créer un fichier PHP à la racine de notre thème, qui va s’appeler checkCaptcha.php et qui aura comme contenu :

<?php
$url = "https://www.google.com/recaptcha/api/siteverify?secret=VOTRE_CLE_PRIVEE&response=".$_GET["response"];
$curl = curl_init($url);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, TRUE);
$output = curl_exec($curl);
curl_close($curl);
echo $output;
?>

Attention : dans le code ci-dessus il faut remplacer VOTRE_CLE_PRIVEE par la valeur correspondante.

Chemin d’accès complet

Il nous faut le chemin d’accès complet chez 1and1. Pour cela on va mettre un fichier temporaire à la racine de notre site qui se nomme info.php dans lequel on inscrit :

<?php
phpinfo();
?>

On accède à cette page via un navigateur pour avoir toutes les informations liées à PHP. On y cherche la ligne qui correspond à DOCUMENT_ROOT afin de trouver le chemin d’accès complet de notre site ; par exemple /kunden/homepages/25/d3506178849/htdocs/clickandbuilds/WordPress/.

Supprimer le fichier info.php.

Création du prepend file

Maintenant on crée un nouveau fichier à la racine de notre site qui se nomme headers.php dans lequel on met :

<?php
$pathinfo=pathinfo($_ENV['SCRIPT_FILENAME']); 
$extension=$pathinfo['extension']; 
switch ($extension) {
  case 'css':
    $contentType = 'text/css';
    break;
  case 'js':
    $contentType = 'application/x-javascript';
    break;
  case 'xml':
    $contentType = 'text/xml';
    break;
  default:
    $contentType = 'text/html';
    break;
}
header("Content-type: ".$contentType);
// pour vérifier que notre script est bien exécuté
header("X-Homemade-Compression: OK");
?>

Ce fichier va être appelé pour chaque page et va permettre d’envoyer le bon Content-Type.

Fichiers .htaccess et php.ini

Maintenant, cela se complique car il va falloir repérer où se situent toutes les ressources que l’on souhaite compresser. Dans le cas de WordPress, il devrait y avoir :

• /wp-includes/js/jquery/*
• /wp-content/themes/votretheme/*

On peut aussi inclure les éventuels répertoires des plugins, et toutes autres ressources.

On se crée un fichier php.ini :

; Server side compression
zlib.output_compression=on
zlib.output_compression_level=9
; tous les fichiers seront parsés par le script headers.php afin d'envoyer le bon Content-Type
auto_prepend_file=/kunden/homepages/25/d3506178849/htdocs/clickandbuilds/WordPress/headers.php

On se crée aussi un fichier .htaccess en y mettant :

AddType x-mapp-php5 .html .htm .css .js .php

Pour que la compression fonctionne sans créer de problème, il va falloir envoyer ces deux fichiers dans chaque répertoire des ressources que l’on souhaite compresser. Par exemple dans /wp-content/themes/votretheme/js/ pour que les fichiers JS du thème soient impactés.

Vous pouvez aussi mettre php.ini à la racine de votre site web, mais sans utiliser le .htaccess de ci-dessus afin d’éviter des problèmes avec l’admin.

On teste

Enfin vous pouvez tester si tout est correct grâce à curl sous Linux :
curl -I -H 'Accept-Encoding: gzip,deflate' --head http://votresite/votreressource.js

Vous devriez voir apparaitre :

X-Homemade-Compression: OK
Content-Encoding: gzip