The solution is to use another end point _api/site/getrecyclebinitems (see the documentation for the various parameters):

fetch("https://tenant.sharepoint.com/sites/YourSite/_api/site/getrecyclebinitems?rowLimit=%2770000%27&itemState=0&orderby=3&isAscending=false", {
  "headers": {
    "accept": "application/json",
    "content-type": "application/json",
    "x-requestdigest": document.querySelector("#__REQUESTDIGEST").value
  },
  "method": "GET",
})
.then(res => res.json())
.then(json => {
  console.log(json.value);
  // it will show an array with an object that contains several interesting properties:
  // AuthorEmail, AuthorName, DeletedByEmail, DeletedDate, DirName, Id, LeafName, Title, …
})

You can then filter the result to get what you need, for example if you’re looking for an item from the list “Projects” that has the ID 1981:

json.value.filter(item => item.DirName.includes("Lists/Projects") && item.LeafName === "1981_.000");

Then, to restore an item, we need the Id from the previous result. The famous endpoint to restore an item is _api/site/RecycleBin('Id')/restore, however it could also return the error “The attempted operation is prohibited because it exceeds the list view threshold”. In that case, we can use this other endpoint _api/site/RecycleBin/RestoreByIds:

fetch("https://tenant.sharepoint.com/sites/YourSite/_api/site/RecycleBin/RestoreByIds", {
  "headers": {
    "accept": "application/json",
    "content-type": "application/json",
    "x-requestdigest": document.querySelector("#__REQUESTDIGEST").value
  },
  "method": "POST",
  "body":JSON.stringify({
    "ids": [
      "4f855ee7-472b-414a-a482-4317a114c1a2" // Id to restore
    ],
    "bRenameExistingItems": true
  })
})

(source)

You’ll need credentials:

• `clientId` – required string, client id obtained when registering the addin
• `clientSecret` – required string, client secret obtained when registering the addin
• `realm` – your SharePoint Online tenant id. The easiest way to find tenant is to open SharePoint Online site collection, click Site Settings → Site App Permissions. Under this page you wll see at least one app “Microsoft.SharePoint”. The tenant id (realm) is highlighted in the image below:
  

Example of the expected result:

{
  clientId: '28bq7e56-8c3a-487d-hbfb-ef1a74539cbe',
  clientSecret: 's6LZ4VvoeKOS+MyAhklcavsyJBF4XhWo06OgY6czYJ0=',
  realm: '85e5f09b-4c17-4d80-afea-260bb171c456'
}

To get the credentials, you need to register a new addin inside SharePoint Online, by fellowing these steps:

1. Open SharePoint Online app registration page, e.g. https://contoso.sharepoint.com/sites/dev/_layouts/15/appregnew.aspx
2. Click on “Generate” for Client id and Client Secret, fill in Title, App Domain, Redirect URI (you can type in any values you want)
   
3. Click on “Create” and save generated Client Id and Client Secret
4. [IF YOU HAVE TENANT RIGHTS] Now you need to apply permissions to the newly registered app. If you want to register the app once and use it for any site collection, it’s better to apply tenant scope permissions, so you can use the credentials everywhere inside your SharePoint tenant. To apply tenant scoped permissions, open AppInv.aspx page under SharePoint adminstration web site, e.g. https://[YOUR_ORGANIZATION]-admin.sharepoint.com/_layouts/15/appinv.aspx, copy paste Client Id from step n°3 into App Id field and click “Lookup”.
5. [IF YOU HAVE TENANT RIGHTS] You will see your registered app, paste in the following XML into the “Permission Request XML” field and click “Create”:

       <AppPermissionRequests AllowAppOnlyPolicy="true">
         <AppPermissionRequest Scope="http://sharepoint/content/tenant" Right="FullControl" />
       </AppPermissionRequests>
   

   

6. [IF YOU ARE NOT A TENANT] If you only want to give permissions on 1 site collection, you can register the app on a regular site collection by using url https://contoso.sharepoint.com/sites/dev/_layouts/15/appinv.aspx. In this case you are not able to use tenant scoped permissions and can only apply site collection permissions:

       <AppPermissionRequests AllowAppOnlyPolicy="true">
         <AppPermissionRequest Scope="http://sharepoint/content/sitecollection" Right="FullControl" />
       </AppPermissionRequests>
   

7. You will see addin “Trust” confirmation, click on “Trust It”:
   
   if trust-it button is not enabled and you get a red label saying tenant admin needs to trust the app, go back and try again in a few minutes.
8. Now you can use client id and client secret to send authenticated http requests.

To know more about the XML permissions, you can check the Microsoft documentation.

Get Access Token

(you can find a C# code as an example)

You need to do a POST request to https://accounts.accesscontrol.windows.net/[YOUR_TENANT_REALM]/tokens/OAuth/2 with a “Content-Type” header that has the value “application/x-www-form-urlencoded”, and the body parameters that must be:

• "grant_type":"client_credentials"
• "client_id":"[YOUR_CLIENT_ID]@[YOUR_TENANT_REALM]"
• "client_secret":"[YOUR_CLIENT_SECRET]"
• "resource":"00000003-0000-0ff1-ce00-000000000000/dell.sharepoint.com@[YOUR_TENANT_REALM]"

See below an example in PHP:

$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, "https://accounts.accesscontrol.windows.net/[YOUR_TENANT_REALM]/tokens/OAuth/2");
curl_setopt($curl, CURLOPT_HTTPHEADER, [ "Content-Type: application/x-www-form-urlencoded" ]);
curl_setopt($curl, CURLOPT_POSTFIELDS, http_build_query([
  "grant_type" => "client_credentials",
  "client_id" => "[YOUR_CLIENT_ID]@[YOUR_TENANT_REALM]",
  "client_secret" => "[YOUR_CLIENT_SECRET]",
  "resource" => "00000003-0000-0ff1-ce00-000000000000/dell.sharepoint.com@[YOUR_TENANT_REALM]"
]));
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
$response = json_decode(curl_exec($curl));
curl_close($curl);

echo $response->access_token;

The response should contain an access token. Example:

{
  "token_type":"Bearer",
  "expires_in":"86399",
  "not_before":"1679393911",
  "expires_on":"1679480611",
  "resource":"00000003-0000-0ff1-ce00-000000000000/dell.sharepoint.com@[YOUR_TENANT_REALM]",
  "access_token":"eyJ0eXAiOiJKV1QiLCJhbGciOiJSU[...]SxXA5Lqbk1OcOVdwQ"
}

Finally, you can do your REST API request to SharePoint Online with passing the header “Authorization” that has the value “Bearer [YOUR_ACCESS_TOKEN]”.

Important warning: when you use PowerApps to manage your form, all edits to the list settings won’t reflect to the PowerApps form. For example, if you add a field, it won’t show up, and you’ll have to manually update the PowerApps to add it (see at the bottom of this article).

From the list view, go to Integrate then PowerApps and Customize forms:

Once PowerApps has open the form, you need to do several things.

1. Load the ID

We first need to make sure the form will load the required item when we pass the ID URL parameter:

From the SharepointForm Advanced Settings, we change the DefaultMode to check if we have the ID parameter, and if we don’t have it, then it should be a New Form, otherwise an Edit Form:

If(IsBlank(Param("ID")), FormMode.New, FormMode.Edit)

From the SharepointForm Advanced Settings, we change the Item section to check if we have the ID parameter, and if we have it, then we do a lookup in our list to find the data:

If(IsBlank(Param("ID")), SharePointIntegration.Selected, LookUp('NAME OF THE LIST', ID = Value(Param("ID"))))

Add a SUBMIT button

With PowerApps, there is no button to save the changes! We’ll add a button in the form:

In the button’s properties, we change the onSelect to be:

SubmitForm(SharePointForm1)

Be aware that the page will stay with the form after clicking on the button. You could want to close using the Exit() function, but the user will be redirected on office.com … I’d recommend to use Launch() by redirecting your users to a page:

SubmitForm(SharePointForm1); Launch("https://tenant.sharepoint.com/sites/MySite/");

Set field’s value based on URL parameter

We can finally set the field’s value based on the parameter in the URL. Select the INPUT zone of the field, and in the Default section we use the below formula:

If(IsBlank(Param("Title")), ThisItem.Title, Param("Title"))

Here my field is called “Title” so I decided to use a parameter called “Title” as well.

Link to the form

We cannot use the NewForm.aspx or EditForm.aspx to access this form, but we need a special link.

Go to your list settings:

Then go to the form settings (it’s from there that you can decide to keep PowerApps or use the original Sharepoint Forms), and click on See versions and usage:

You’ll get the App Id from this page:

Next, you’ll use the App Id to forge the URL: https://apps.powerapps.com/play/providers/Microsoft.PowerApps/apps/APP_ID
With our example, the URL will be: https://apps.powerapps.com/play/providers/Microsoft.PowerApps/apps/c6f23ac1-dcbd-4daf-925e-2701ab241ca0

You can now pass the URL parameter: https://apps.powerapps.com/play/providers/Microsoft.PowerApps/apps/APP_ID?Title=Hello%20World
And an ID to retrieve an existing item: https://apps.powerapps.com/play/providers/Microsoft.PowerApps/apps/APP_ID?Title=Hello%20World&ID=2

How to use it with a LookUp column?

If you want to auto-select a LookUp field using an URL parameter, you need to do a few things…

First, we need to add the related table. To do so, click on Data in the left navigation bar and search for SharePoint:

Search for the table and add it.

Second (optional) step: click on the Lookup field in the form and change the Items to show a list of options – if no “Lookup” ID in the URL, then we use the default list of options:

The below formula permits to retrieve the “ID” and “Title” from the distant list, base on the “Lookup” parameter, and to rename the result as {Id:"ID", Value:"Title"}:

If(IsBlank(Param("Lookup")), Choices([@'CURRENT LIST NAME'].COLUMN_NAME), RenameColumns(ShowColumns(Filter('DISTANT LIST NAME', ID = Value(Param("Lookup"))), "ID", "Title"), "ID", "Id", "Title", "Value"))

Third, click on the Lookup field in the form and change the DefaultSelectedItems to select the item from the list of options:

The below formula returns an empty selection with {Id:"", Value:""} when no URL param, otherwise it returns the first record for our lookup:

If(IsBlank(Param("Lookup")), {Id:"", Value:""}, First(RenameColumns(ShowColumns(Filter('DISTANT LIST NAME', ID = Value(Param("Lookup"))), "ID", "Title"), "ID", "Id", "Title", "Value")))

And finally, we can pass Lookup=ID in the URL to select the related item in the other list

How to deal with new fields?

If you add a new field to your list’s settings, you’ll have to edit the form in PowerApps, and then edit the fields and add the new one:

(I used this article as a starting point)

A colleague has an older version (april 2020) that works correctly. Here is the link to download this version: SharepointMigrationToolSetup_April2020

Add your website to the Trusted Website zone:

Next open Windows Explorer and right-click on “Your PC “, then select “Map network drive…”:

You need to use a special path to your website. Let’s say your website is located at https://tenant.sharepoint.com/sites/Marketing then the path to use is:
\\tenant.sharepoint.com@ssl\DavWWWRoot\sites\Marketing\

Note the @ssl and DavWWWRoot that are required.

Assign this path to a drive letter:

Hopefully some of the operations can be done with REST API, and in my organization the threshold limitation is removed during the night (1am to 4am). By combining a schedule service (like node-schedule) with my library SharepointPlus I can run this admin tasks during the night while my virtual computer is running.

To delete a column

$SP().ajax({
  url:"https://company.com/sharepoint/team/sales/_api/web/lists/getbytitle('My List')/fields/getbytitle('Column To Delete')",
  method: "POST",
  headers: {
    "IF-MATCH": "*",
    "X-HTTP-Method": "DELETE"
  }
});

To index a column

$SP().ajax({
  url:"https://company.com/sharepoint/team/sales/_api/web/lists/getbytitle('My List')/fields/getbytitle('Column to Index')",
  method: "POST",
  body:JSON.stringify({
    "__metadata":{ type: "SP.Field" },
    "Indexed":true
  }),
  headers: {
    "IF-MATCH": "*",
    "X-HTTP-Method": "MERGE"
  }
})

To delete a list

$SP().ajax({
  url:"https://company.com/sharepoint/team/sales/_api/web/lists/getbytitle('List To Delete')",
  method: "POST",
  headers: {
    "IF-MATCH": "*",
    "X-HTTP-Method": "DELETE"
  }
})

To delete a list

$SP().ajax({
  url:"https://company.com/sharepoint/team/sales/_api/web/lists/getbytitle('List To Delete')",
  method: "POST",
  headers: {
    "IF-MATCH": "*",
    "X-HTTP-Method": "DELETE"
  }
})

To delete a site

$SP().ajax({
  url:"https://company.com/sharepoint/team/sales/website_to_delete/_api/web/",
  method: "POST",
  headers: {
    "X-HTTP-Method": "DELETE"
  }
})

Location of the Search Rest service

The Search REST service is located at the following URI: http://host/site/_api/search

The Search REST API

The Search REST service exposed by the URI specified above provides the following methods or entry points:

Query Parameters

The following table lists the query parameters available for the query entry point:

Suggest Parameters

The following table lists the query parameters available from the suggest entry point:

 References:

• [MS-SRCHCSOM]: Search Client Query Protocol Specification
• Search in SharePoint 2013 on MSDN

Examples

A postquery request will look like:

POST http://localhost/_api/search/postquery HTTP/1.1
Accept: application/json;odata=verbose
Content-Type: application/json;odata=verbose
Host: localhost
Content-Length: 419

{

 'request': {
     'Querytext': 'sharepoint',
     'StartRow':1,
     'RowLimit':8,
     'SelectProperties': {
          'results': ['Title','ContentSource','DisplayAuthor']
     },
     'TrimDuplicates':true,
     'Refiners':'companies,contentclass,ContentType(filter=7/0/*),FileExtension(filter=7/0/*),FileType(filter=6/0/*)',
     'RefinementFilters': {'results': ['filetype:equals("pptx")'] }
  }
}

“The e-mail message cannot be sent. Make sure the e-mail has a valid recipient.”

After searching the web, I found the solution: instead of using an email address for the recipients you have to use the userDomain with “i:0#.w|”.

For example, if the user is John Doe, and his email is john_doe@company.com, then you should use something like i:0#.w|domain\\john_doe.

DateRangesOverlap Value Type

I don’t know if any back-end developers have run into this same issue when working with the SPQuery to pull expanded recurrence date, but from here, not all DateRangesOverlap Value types work the way they should. And they’re cryptically-documented, where mentioned at all. The basics are that you can return dates that “overlap” a given scenario. The overlapping varies depending on scenario, as follows:

<Year />
This is supposed to get all events within:
(a) Today’s year (if no CalendarDate is passed), or
(b) the date passed in QueryOptions as CalendarDate.

Actually, it returns all dates from [Today] into the future. It ignores:
(a) the CalendarDate, and
(b) the end of the current year

<Month />
Performs consistently as expected, with the exception that its definition of “month” is more of a “month view.” A month view consists of all of the weeks which contain days in that month. In other words, the month view for May 2012 begins on April 29, 2012, because May 1 is on a Tuesday and the “month view” consists of full weeks. Likewise, the May 2012 month view ends on June 5, 2012. This means that pulling all of the dates for May 2012 will return all events for April 29 – June 5.

However, even though April 29 is the beginning of the May month view, when passed as the CalendarDate, it will return the April month view. See below for how this affects date calculations based on pulling data for a given month.

<Week />
Performs exactly like <Month />, but only returns the corresponding week for:
(a) Today, or
(b) CalendarDate

<Today />
Returns just the events for Today.

<Now />
Contrary to most documentation, <Now /> doesn’t “work” but rather defaults to the same results as <Year />.

In fact, <Year /> is the default. If you leave off the entire WHERE clause, but include the RecurrenceDate and fRecurrence fields and the QueryOptions (CalendarDate will be ignored), you get the same results as <Year />.

CalendarDate

In QueryOptions, if the CalendarDate is set, it needs the date to be a string in the format:
yyyy-mm-ddThh:mm:ssZ

In most cases, setting the CalendarDatewhen needed to the real date on which you want to base your range will work just fine, as long as you set the time in the Z-notation to noon. The reason for the noontime setting is to make allowance for all-day events within the given range. These tend to get messed up when viewing is based on a midnight or early morning time. Basing the range on noon alleviates the issue. (At least it does for US time zones; I confess that I haven’t tested them all.)

This protection can be useful, but also very annoying, for example when the file is not closed properly, then the lock could stay “forever”.

There are many posts on the web about it.

I found one that has been super useful: https://pholpar.wordpress.com/2014/04/07/how-to-use-javascript-to-delete-short-term-locks-from-documents-opened-from-sharepoint/
The author explains very well the different steps what I’m trying to summarize:

1. Send a request to _vti_bin/_vti_aut/author.dll with special headers/body
2. Auth.dll will provide the lockid
3. Send a request to _vti_bin/cellstorage.svc/CellStorageService with special headers/body, included the lockid
4. The file is unlocked

The code to send to CellStorageService, and provided by the author, didn’t work for me. I’ve had to use Fiddler and open the document into Office on my computer to see the kind of requests send by it to unlock a document. Based on it, I’ve re-built the code and you can find my solution below.

Tested on Sharepoint 2013 On-Promise only. I don’t know if this solution works for Sharepoint Online or other version.
Also note that I use $SP().ajax() from SharepointPlus, but it’s equivalent to the $.ajax from jQuery.

// full path to the document
var docUrl = "https://website.com/website/Doc_Library/Test.docx";

// start by querying author.dll to find the lockid and the user who locked it
$SP().ajax({
  url: 'https://website.com/website/_vti_bin/_vti_aut/author.dll',
  headers:{
    "Content-Type": "application/x-www-form-urlencoded",
    "MIME-Version": "1.0",
    "Accept": "auth/sicily",
    "X-Vermeer-Content-Type": "application/x-www-form-urlencoded"
  },
  body: 'method=getDocsMetaInfo%3a14%2e0%2e0%2e6009&url%5flist=%5b' + encodeURIComponent(docUrl) + '%5d&listHiddenDocs=false&listLinkInfo=false',
}).then(function(source) {
  // go thru the source page returned to find the lockid and current user
  var nextLine = false;
  var ret = { "lockid":"", "user":"", when:"" };
  source.split("\n").forEach(function(line) {
    if (line.indexOf("vti_sourcecontrollockid") !== -1) nextLine="lockid"; // vti_sourcecontrollockid -> the lockid to use later
    else if (line.indexOf("vti_sourcecontrolcheckedoutby") !== -1) nextLine="user"; // vti_sourcecontrolcheckedoutby -> username of the user who locked it
    else if (line.indexOf("vti_sourcecontrollockexpires") !== -1) nextLine="when"; // vti_sourcecontrollockexpires -> when the server is supposed to unlock it
    else if (nextLine !== false) {
      ret[nextLine] = line.slice(7).replace(/&#([0-9]|[1-9][0-9]|[[01][0-9][0-9]|2[0-4][0-9]|25[0-5]);/g, function (str, match) { return  String.fromCharCode(match); });
      nextLine = false;
    }
  });

  if (!ret.lockid) { alert("Not Locked") }
  else {
    // compose a request based on what Microsoft Office sends to the Sharepoint server
    // found using Fiddler
    var releaseLockReq = '<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"><s:Body><RequestVersion Version="2" MinorVersion="2" xmlns="http://schemas.microsoft.com/sharepoint/soap/"/><RequestCollection CorrelationId="{96A244BD-D13B-4696-9355-231FB673BC4F}" xmlns="http://schemas.microsoft.com/sharepoint/soap/"><Request Url="'+docUrl+'" UseResourceID="true" UserAgent="{1984108C-4B93-4EEB-B320-919432D6E593}" UserAgentClient="msword" UserAgentPlatform="win" Build="16.0.8201.2102" MetaData="1031" RequestToken="1"><SubRequest Type="ExclusiveLock" SubRequestToken="1"><SubRequestData ExclusiveLockRequestType="ReleaseLock" ExclusiveLockID="'+ret.lockid+'"/></SubRequest></Request></RequestCollection></s:Body></s:Envelope>';

    // we send it to the webservice cellstorage.svc
    $SP().ajax({
      url:'https://website.com/website/_vti_bin/cellstorage.svc/CellStorageService',
      body:releaseLockReq,
      headers:{
        'Content-Type':'text/xml; charset=UTF-8',
        'SOAPAction': "http://schemas.microsoft.com/sharepoint/soap/ICellStorages/ExecuteCellStorageRequest"
      }
    })
    .then(function(res) {
      if (res.indexOf('ErrorCode="Success"') !== -1) alert("Success") // the file has been unlocked
      else alert("Failed")
    })
  }
})

I hope it will be useful to someone else!

However you cannot use the same trick to preview a file into the browser with WopiFrame.aspx. But there is a way to do it, using the parameters into the URL.

Let’s say the url to preview your document is:
https://my.company.com/sharepoint/_layouts/15/WopiFrame.aspx?sourcedoc=/files/Demo.docx&action=default

You need to replace action=default with action=embedview, AND you need to call WopiFrame2.aspx instead of WopiFrame.aspx (which will lead to a Something Went Wrong - Sorry, you don't have access to this page).

So the URL to use for your iframe will be:
https://my.company.com/sharepoint/_layouts/15/WopiFrame2.aspx?sourcedoc=/files/Demo.docx&action=embedview

Below is a way to do it by using jQuery and SharepointPlus. It will hide the “Specify your own value” stuff, and add the old value (if any) into the dropdown selection.

For example if you have a Choice field with options “A”, “B”, “C”. Your item ID #1 has value “B”.
After a while you decide to delete “B” and add “D”, but you want to be able to find items with the “B” value.
So you choose “Choice with Fill-In” and apply the below code:

// For flexibility reasons we create them as a Choice with Fill-In option,
//    however we hide the free text field and we show only the dropdown
//    in case of the fill-in has a value, then we add it into the list of options
$SP().formfields(["My Fist Dropdown", "Another One"]).each(function() {
  var $e = this.elem();
  // hide all except the dropdown
  //$e.not('select').hide().filter(':radio:last').closest('tr').hide(); // SP2010
  $e.eq(0).closest('table').find('tr:gt(0)').hide(); // SP2013
  // if we have a value into the fill-in box, then:
  //   - add it into the options
  //   - when another value is selected we check the first checkbox
  var fillValue = $e.last().val();
  if (fillValue) {
    $e.filter('select').append('<option data-fillin="true">'+fillValue+'</option>').each(function() {
      var $this=$(this);
      $this.find('option:last').prop("selected", true);
      $this.removeAttr("onclick").on('change', function(event) {
        var $opt = $(this).find(':selected');
        if ($opt.data("fillin")) {
          $e.filter(':radio:last').prop("checked", true);
        } else {
          $e.filter(':radio:first').prop("checked", true);
        }
      });
    });
  }
})

So now, if you create a new item you’ll see a dropdown with “A”, “C”, and “D” only.
But if you edit your item #1, you’ll have a dropdown with “A”, “C”, “D” and “B”.

// we need to make sure sp.core.js is loaded
SP.SOD.executeOrDelayUntilScriptLoaded(function() {
  window.UploadFinishFunc=function(b, a) {
    typeof g_currentControl.postUploadFunc == "function" && a.status != UploadStatus.CANCELLED && g_currentControl.postUploadFunc(a.files);
    a.status != UploadStatus.CANCELLED && myPostUploadFunc(a);
    g_currentControl.status = ControlStatus.UPLOADED;
    UpdateProgressBar(ProgressMessage.UPLOADED, a);
    RefreshResult(a);
    g_currentControl.status = ControlStatus.IDLE
  }
}, 'sp.core.js');
SP.SOD.executeFunc('sp.core.js')

// below is the function that will be called
function myPostUploadFunc(data) {
  console.log("myPostUploadFunc => ",data)
}

Here is the result:

// source: https://www.eliostruyf.com/adding-a-custom-action-to-a-callout-in-sharepoint-2013/
// add a special callout action for our library
SP.SOD.executeFunc("callout.js", "Callout", function () {
  var itemCtx = {};
  itemCtx.Templates = {};
  itemCtx.BaseViewID = 'Callout';
  // Define the list template type
  itemCtx.ListTemplateType = 101;
  itemCtx.Templates.Footer = function (itemCtx) {
    // context, custom action function, show the ECB menu (boolean)
    return CalloutRenderFooterTemplate(itemCtx, AddCustomAction, true);
  };
  SPClientTemplates.TemplateManager.RegisterTemplateOverrides(itemCtx);
});

function AddCustomAction (renderCtx, calloutActionMenu) {
  var itemIndex = renderCtx.CurrentItemIdx
  // Add your custom action
  calloutActionMenu.addAction(new CalloutAction ({
    text: "Custom Action",
    tooltip: 'This is your custom action',
    onClickCallback: function() {
      // all the data related to your item are into `renderCtx.ListData.Row[itemIndex]`
      console.log('Callback from custom action');
    }
  }));
  // re-add EDIT action
  calloutActionMenu.addAction(new CalloutAction ({
    text: "Edit",
    onClickCallback: function(event) {
      // use the default action we have when clicking on the filename into the library
      // or call the EditForm if it's a list item and not a library
      DispEx(this, event,'TRUE','FALSE','FALSE','SharePoint.OpenDocuments.3','1','SharePoint.OpenDocuments','','1https://your.sharepoint.com/_layouts/15/WopiFrame.aspx?sourcedoc='+renderCtx.ListData.Row[itemIndex].FileRef+'&action=default','','1','0','0','0x7fffffffffffffff')
    }
  }));
}

It’s actually pretty simple. You need to add the below tag into your .aspx page:

<asp:Content ContentPlaceHolderID="PlaceHolderLeftNavBar" runat="Server"></asp:Content>

Tested with Sharepoint 2013 On-Promise.

@font-face {
  font-family: 'Roboto';
  src: url('/_catalogs/masterpage/css/fonts/Roboto/Regular/Roboto-Regular.woff2') format('woff2'),
       url('/_catalogs/masterpage/css/fonts/Roboto/Regular/Roboto-Regular.woff') format('woff'),
       url('/_catalogs/masterpage/css/fonts/Roboto/Regular/Roboto-Regular.ttf') format('truetype');
  font-weight: 400;
  font-style: normal;
}

To have this resource sent back from the Sharepoint server with cache headers, you just need to add ?ctag=0 at the end:

@font-face {
  font-family: 'Roboto';
  src: url('/_catalogs/masterpage/css/fonts/Roboto/Regular/Roboto-Regular.woff2?ctag=0') format('woff2'),
       url('/_catalogs/masterpage/css/fonts/Roboto/Regular/Roboto-Regular.woff?ctag=0') format('woff'),
       url('/_catalogs/masterpage/css/fonts/Roboto/Regular/Roboto-Regular.ttf?ctag=0') format('truetype');
  font-weight: 400;
  font-style: normal;
}

You first need to Create a new view named Unpublished Content. You can sort by Name, and scroll down to the Folders settings and choose Show all items without folders.

We now have all our files in the view.

Next, switch to the Library ribbon and choose Modify in SharePoint Designer (Advanced) from the Modify View dropdown.

Sharepoint Designer will open your page.

In Sharepoint Designer, switch to the Home tab and choose Advanced Mode. The file is reloaded.

Search into your code the tag ListViewXml. On this line you should have &lt;Query;&&gt;. Just after it, we need to add:
<Where><Or><Eq><FieldRef Name="_ModerationStatus" /><Value Type="ModStat">Draft</Value></Eq><IsNotNull><FieldRef Name="CheckoutUser" /></IsNotNull></Or></Where>

Before:

After:
(click to enlarge)

Save, and your view should only show the unpublished content!

(Article inspired by https://thechriskent.com/2013/02/14/unpublished-view/)

To verify the PermMask against the different level of permissions you can proceed with the below method (using functions from core.js):

var permMask = '0x1b03c4313ff';
var permissions = new SP.BasePermissions();
permissions.initPropertiesFromJson({High:GetPermMaskH(permMask), Low:GetPermMaskL(permMask)});
// we can now check permissions using SP.BasePermissions.has()
// and we can compare with SP.PermissionKind — see https://msdn.microsoft.com/en-us/library/office/ee556747(v=office.14).aspx
var canEdit = permissions.has(SP.PermissionKind.editListItems);

On Sharepoint 2010 GetPermMaskH and GetPermMaskL are not defined, so here is their code:

function GetPermMaskH(b) {
    var a = b.length;
    return a <= 10 ? 0 : parseInt(b.substring(2, a - 8), 16)
}

function GetPermMaskL(b) {
    var a = b.length;
    return a <= 10 ? parseInt(b) : parseInt(b.substring(a - 8, a), 16)
}

It took me many hours to find how to proceed, so I hope it will help some others.

If you want to decode those unicode strings, you can use this short code:
unescape(JSON.parse('"' + s.replace('"', '\\"') + '"'))

To change it I used the below Javascript code:

// replace the Delete Item default action when into a popup
$(document).ready(function() {
  // if it's in a popup
  if (window !== window.top) {
    $('a[id$="_toolBarTbl_RptControls_diidIODeleteItem"]').attr("href","javascript:commonDeleteItem(); return false")
  }
})
// now when clicking on the Delete item it will call my own function
// here I use http://aymkdn.github.io/SharepointPlus/ to show a waiting message and to delete the current item, and then close the modal
function commonDeleteItem() {
  $SP().waitModalDialog("Deleting...");
  $SP().list("{"+WPQ2FormCtx.ListAttributes.Id+"}").remove({ID:GetUrlKeyValue("ID")}, {
    after:function() {
      $SP().closeModalDialog();
      $SP().closeModalDialog(2);
    }
  })
}

After searching a bit about it, I found a clue that helped me to find a good solution: on Firefox/Chrome when you click on an Excel file, you’ll now have a modal asking if you want to open it in edit or readonly mode!

To achieve this result you have to insert the below Javascript code somewhere into your masterpage:

/**
 * fix problem with Excel documents on Firefox/Chrome
 * @param  {HTMLElement} p the <A> element
 * @param  {HTMLEvent} a the click event
 * @param  {Boolean} h TRUE
 * @param  {Boolean} e FALSE
 * @param  {Boolean} g FALSE
 * @param  {Strin} k the ActiveX command (e.g. "SharePoint.OpenDocuments.3")
 * @param  {Number} c 0
 * @param  {String} o the activeX command, here we look at "SharePoint.OpenDocuments"
 * @param  {String} m
 * @param  {String} b the replacement URL to the xslviewer
 */
var bak_DispEx;
var modalOpenDocument; // it will be use with the modal
SP.SOD.executeOrDelayUntilEventNotified(function() {
  bak_DispEx = _DispEx;
  _DispEx=function(p, a, h, e, g, k, c, o, m, b, j, l, i, f, d) {
    // if o==="SharePoint.OpenDocuments" && !IsClientAppInstalled(o)
    // in that case we want to open ask the user if he/she wants to readonly or edit the file
    var fileURL = b.replace(/.*_layouts\/xlviewer\.aspx\?id=(.*)/, "$1");
    if (o === "SharePoint.OpenDocuments" && !IsClientAppInstalled(o) && /\.xlsx?$/.test(fileURL)) {
      // if the URL doesn't start with http
      if (!/^http/.test(fileURL)) {
        fileURL = window.location.protocol + "//" + window.location.host + fileURL;
      }
      var ohtml = document.createElement('div');
      ohtml.style.padding = "10px";
      ohtml.style.display = "inline-block";
      ohtml.style.width = "200px";
      ohtml.style.width = "200px";
      ohtml.innerHTML = '<style>'
                      + '.opendocument_button { background-color:#fdfdfd; border:1px solid #ababab; color:#444; display:inline-block; padding: 7px 10px; }'
                      + '.opendocument_button:hover { box-shadow: none }'
                      + '#opendocument_readonly,#opendocument_edit { float:none; font-size: 100%; line-height: 1.15; margin: 0; overflow: visible; box-sizing: border-box; padding: 0; height:auto }'
                      + '.opendocument_ul { list-style-type:none;margin-top:10px;margin-bottom:10px;padding-top:0;padding-bottom:0 }'
                      + '</style>'
                      + 'You are about to open:'
                      + '<ul class="opendocument_ul">'
                      + '  <li>Name: <b>'+fileURL.split("/").slice(-1)+'</b></li>'
                      + '  <li>From: <b>'+window.location.hostname+'</b></li>'
                      + '</ul>'
                      + 'How would like to open this file?'
                      + '<ul class="opendocument_ul">'
                      + '  <li><label><input type="radio" name="opendocument_choices" id="opendocument_readonly" checked> Read Only</label></li>'
                      + '  <li><label><input type="radio" name="opendocument_choices" id="opendocument_edit"> Edit</label></li>'
                      + '</ul>'
                      + '<div style="text-align: center;margin-top: 20px;"><button type="button" class="opendocument_button" style="background-color: #2d9f2d;color: #fff;" onclick="modalOpenDocument.close(document.getElementById(\'opendocument_edit\').checked)">Open</button> <button type="button" class="opendocument_button" style="margin-left:10px" onclick="modalOpenDocument.close(-1)">Cancel</button></div>';
      // show the modal
      modalOpenDocument=SP.UI.ModalDialog.showModalDialog({
        html:ohtml,
        dialogReturnValueCallback:function(ret) {
          if (ret!==-1) {
            if (ret === true) { // edit
              // reformat the fileURL
              var ext;
              if (/\.xlsx?$/.test(b)) ext = "ms-excel";
              if (/\.docx?$/.test(b)) ext = "ms-word"; // not currently supported
              fileURL = ext + ":ofe|u|" + fileURL;
            }
            window.location.href = fileURL; // open the file
          }
        }
      });
      a.preventDefault();
      a.stopImmediatePropagation()
      a.cancelBubble = true;
      a.returnValue = false;
      return false;
    }
    return bak_DispEx.apply(this, arguments);
  }
}, "sp.scriptloaded-core.js")

To use the paging option and get all the content from a large Sharepoint list you can simply do the below code:

$SP().list("My List").get({
  fields:"ID,Title",
  rowlimit:5000,
  paging:true,
  progress:function progress(nbItemsLoaded) {
    // for each new page this function will be called
    console.log("It's still loading... already "+nbItemsLoaded+" items have been loaded!");
  }
}, function(data) {
  console.log(data.length); // -> 23587
  for (var i=0, len=data.length; i<len; i++) {
    console.log(data[i].getAttribute("ID"))
  }
})

If you only want to get the first 15,000 items, you can use the option page:3. See the documentation to know more.

// this function will do our asynchronous check
function beforeSaving() {
  var deferred=jQuery.Deferred();
  setTimeout(function() {
    alert("Test Complete!")
    deferred.reject(); // if the test failed
    // or deferred.resolve() to valid the test
  }, 2000)
  return deferred;
}
// force PostBackRequired to true in the context, otherwise it won't work if you open it into a modal
WPQ2FormCtx.PostBackRequired=true;
// we override SPClientForms.ClientFormManager.SubmitClientForm
SPClientForms.ClientFormManager.SubmitClientForm=function(b){
  var a=SPClientForms.ClientFormManager.GetClientForm(b);
  var res = (a!=null&&a.SubmitClientForm());
  // if the form is not valid, then res===true
  if (res === true) return true;
  else {
    // at this stage, all fields' value have been saved into a hidden input
    // e.g. document.querySelector('input[type="hidden"][id$="Field_x0020_Name"]')
    // all these saved values will be sent back to the server when "WebForm_DoPostBackWithOptions" is executed

    // if the form is valid we now want to do our asynchronous check
    beforeSaving().done(function() {
      // our test is valid too so we can send the form to the server
      WebForm_DoPostBackWithOptions(new WebForm_PostBackOptions($get(WPQ2FormCtx.SubmitButtonID).name, "", true, "", "", false, true))
    }).fail(function() {
      // if it fails we just unblock the save button
      $get(WPQ2FormCtx.SubmitButtonID).disabled=false;
    })
  }
  // to make sure the form won't be submitted yet
  return true;
};

You can also add it into your masterpage to allow it everywhere on your website. Add it into your <head> block:

Now you can use postMessage() to communicate with your iframe.

To do so we will call our file using the JSLink (see my previous post about it).
The JS file looks like that:

(function() {
  // do some actions as soon as the fields are shown
  var loadAfterForm = {
    Templates: {
      OnPostRender:function(ctx) {
        // hide the field/row when "[INTERNAL USE - DO NOT CHANGE]" is detected in the field's description
        if (ctx.ListSchema.Field[0].Description.indexOf('[INTERNAL USE - DO NOT CHANGE]') > -1) {
          // get the element that represents the field
          var elem = document.querySelector('[id^="'+ctx.ListSchema.Field[0].Name +'_'+ ctx.ListSchema.Field[0].Id+'"]');
          while (elem) {
            elem = elem.parentNode;
            if (elem) {
              if (elem.tagName==="BODY") break;
              if (elem.tagName === "TD" && elem.className.indexOf("ms-formbody") > -1) {
                // hide the whole row
                elem.parentNode.style.display="none";
                break;
              }
            }
          }
        }
      }
    }
  }
  SPClientTemplates.TemplateManager.RegisterTemplateOverrides(loadAfterForm);
})();

Sharepoint is really annoying to limit the access to the list with more than 5,000 items… It causes many troubles, and one of them is when you use a lookup field into a form that is tied to a big list. Sharepoint will not display the field/dropdown but will show the message:

This is a lookup column that displays data from another list that currently exceeds the List View Threshold defined by the administrator

There is a solution to bypass this problem and show our lookup dropdowns.

You need:

• jQuery
• Sharepoint
• And I use Chosen to have a better dropdown

I’ll show how I did it for the EditForm.

First, copy/paste the below JavaScript code into a file that you’ll store somewhere on your Sharepoint:

// load jQuery, SharepointPlus and Chosen
var toLoad = '<script src="/Toolbox/Documents/js/jQuery/jquery-1.12.3.min.js"></script>'
           + '<script src="/Toolbox/Documents/js/SharepointPlus/3.13/sharepointplus-3.13.min.js"></script>'
           + '<link href="/Toolbox/Documents/js/Chosen/1.5.0/chosen.min.css" rel="stylesheet" type="text/css">'
           + '<script src="/Toolbox/Documents/js/Chosen/1.5.0/chosen.jquery.min.js"></script>'
document.write(toLoad);

(function() {
  var aDeferred=[];
  var modal;
  // list the fields we want to fix
  // /!\ Add here the Field ID of the fields that need to be fixed
  var lookupFieldsToFix = ["Voucher_x0020_Code", "User_x0020_Name"];
  // /!\ Specify the name of the related list, as well as the Column to retrieve
  var configurationsLookup = [{list:"Vouchers", field:"Title"}, {list:"Users", field:"Name"}];

  // this function is triggered once the form is loaded
  function allLoaded(ctx) {
    // delete all existing localStorage
    localStorage.removeItem('RequestsLookup')
    // check if our call to lists are done
    $.when.apply(this, aDeferred).done(function() {
      var save={}, count=0, i;
      for (i=arguments.length; i--;) {
        if (arguments[i]) {
          save[arguments[i].field] = arguments[i].choices; // save it as {fieldName:choices}
          count++;
        }
      }
      if (count > 0) {
        // we use localStorage
        localStorage.setItem('RequestsLookup', JSON.stringify(save));
        // now reload the page
        $SP().closeModalDialog();
        $('#aspnetForm').hide().after('<h1>Reloading...</h1>');
        window.location.reload();
      } else {
        // we use Chosen
        for (i=lookupFieldsToFix.length; i--;) {
          var e=WPQ2FormCtx.ListSchema[lookupFieldsToFix[i]];
          $(document.getElementById(e.Name+"_"+e.Id+"_$"+e.FieldType+"Field")).chosen({search_contains:true});
        }
        // and on Exam Name
        $SP().formfields("Exam Name").elem().chosen({search_contains:true});
      }
    })
  }
  
  /**
   * Fix the broken lookup fields
   * @param  {String} field Name of the field to fix
   * @return {Deferred}
   */  
  function fixLookup(field) {
    var deferred = jQuery.Deferred();
    var saved, choices=[], config;

    // we check if there is a Throttled
    if (WPQ2FormCtx.ListSchema[field].Throttled) {
      WPQ2FormCtx.ListSchema[field].Throttled=false;
      // check if we have a localStorage, if yes it means we are after the reload
      saved = localStorage.getItem("RequestsLookup");
      if (saved !== null) {
        saved = JSON.parse(saved);
        // we use the stored data to create our dropdown
        WPQ2FormCtx.ListSchema[field].ChoiceCount=saved[field].length;
        WPQ2FormCtx.ListSchema[field].Choices=saved[field];
        deferred.resolve();
      } else {
        WPQ2FormCtx.ListSchema[field].ChoiceCount=0;
        WPQ2FormCtx.ListSchema[field].Choices=[];
        // then we show a Waiting message
        if (!modal) {
          modal=true;
          $SP().waitModalDialog("Loading some data...");
        }
        // and we get data from the list
        config = configurationsLookup[SPArrayIndexOf(lookupFieldsToFix, field)];
        if (config.list) {
          $SP().list(config.list).get({fields:config.field, paging:true}, function(data) {
            var res=[];
            for (var i=data.length; i--;) {
              res.push({LookupId:data[i].getAttribute("ID"), LookupValue:data[i].getAttribute(config.field)});
            }
            deferred.resolve({field:field, choices:res});
          });
        } else deferred.resolve([]);
      }

    } else deferred.resolve();
    return deferred;
  }

  // do some actions as soon as the fields are shown
  var changeForm = {
    Templates: {
      OnPreRender:function(ctx) {
        // we want to show Voucher Code and User Name even if there are more than 5000 items in those lists
        if (SPArrayIndexOf(lookupFieldsToFix, ctx.ListSchema.Field[0].Name) > -1) {
          aDeferred.push(fixLookup(ctx.ListSchema.Field[0].Name));
        }
      },
      OnPostRender:function(ctx) {
        // only trigger when everything is loaded
        if (ctx.ListSchema.Field[0].Name === "Attachments") {
          allLoaded(ctx)
        }
      }
    }
  }
  // don't do it when editing the page
  if (GetUrlKeyValue("PageView") !== "Shared") SPClientTemplates.TemplateManager.RegisterTemplateOverrides(changeForm);
})();

Please refer to the /!\ symbol to configure two variables: only lookupFieldsToFix and configurationsLookup need to be set by you.
Note that the above code is compatible for IE8+, and all modern browsers.

Now go to the EditForm.aspx of your list. Then edit the page:

Next, edit the webpart settings:

Go to the Miscellaneous section and enter the path to your JavaScript file (created before) in the JS Link field – you can use ~site in the path:

Finally click ON, and then on Stop Editing in the ribbon.

You can now try to edit an item and the lookup fields should be fixed. Example:

NOTE

The document.write() is not really a good practice because it will delay the page load. It would be better to load asyn the JS using this tiny function. In that case you need to make sure to have only Vanilla JS in your changeForm.

My attempt here is to remove some rows from the DispForm. To do so I needed to trigger an action as soon as the fields rendering has been done. After different tries, I finally came up with the PostRender option.

// https://gist.github.com/Aymkdn/98acfbb46fbe7c1f00cdd3c753520ea8
function loadExt(e,t){var s=this;s.files=e,s.js=[],s.head=document.getElementsByTagName("head")[0],s.after=t||function(){},s.loadStyle=function(e){var t=document.createElement("link");t.rel="stylesheet",t.type="text/css",t.href=e,s.head.appendChild(t)},s.loadScript=function(e){var t=document.createElement("script");t.type="text/javascript",t.src=s.js[e];var a=function(){++e<s.js.length?s.loadScript(e):s.after()};t.onload=function(){a()},s.head.appendChild(t)};for(var a=0;a<s.files.length;a++)/\.js$|\.js\?/.test(s.files[a])&&s.js.push(s.files[a]),/\.css$|\.css\?/.test(s.files[a])&&s.loadStyle(s.files[a]);s.js.length>0?s.loadScript(0):s.after()}

// verify when all scripts have been loaded
var loadExtLoaded = false;
function checkExt(ctx) {
  if (loadExtLoaded) {
    // here you can call a function that is in one of the called script
  } else {
    setTimeout(function() { checkExt(ctx) }, 50);
  }
}
loadExt([
  'file_you_want_to_load.js',
  'style_you_want_to_load.css',
  'no_cache_file.js?timestamp='+(Date.now())
], function() {
  loadExtLoaded=true;
});

(function() {
  function onLoad(ctx) {
    checkExt(ctx);
  }

  // do some actions as soon as the fields are shown
  var loadAfterForm = {
    Templates: {
      OnPostRender:function(ctx) {
         // only trigger when everything is loaded
        // --> ctx.ListData.Items[0] all the fields
        if (ctx.ListSchema.Field[0].Name === "Attachments") {
          onLoad(ctx)
        }
      }
    }
  }
  SPClientTemplates.TemplateManager.RegisterTemplateOverrides(loadAfterForm);
})();