load-balancingumbraco8examine

Umbraco Examine File Locking Azure App Service


We are running an Umbraco 8.1.5 website in a series of load balanced Azure App Services and are having Examine/Lucene file locking issues in one of the instances. We have tried upgrading Examine to 1.0.2 but this still hasn't helped.

From what I can see we have followed all the documentation for setting up Umbraco to run in an Azure App Service load balanced environment.

Has anyone got load balancing working with Umbraco 8 in Azure App Services that are scaled/auto-scaling?

App Services

Admin - one instance, no auto scaling Front-end - scaled to two instances

On one of the front-end servers our search controller that accesses the external index is working but on the other instance it is erroring saying that the index files are locked. Error messages have been provided below.

If we scale in the front-end to a single instance search works all the time.

Setup

Server roles

Admin web.config App Setting

<add key="UmbracoServerRegistrar" value="master" />

Front-end web.config App Setting

<add key="UmbracoServerRegistrar" value="slave" />

This is picked up in the following composer

public class ServerRegistrarComposer : IUserComposer
{
    public void Compose(Composition composition)
    {
        var config = System.Configuration.ConfigurationManager.AppSettings["UmbracoServerRegistrar"];
        if (config != null && config.Equals("slave", StringComparison.InvariantCultureIgnoreCase))
        {
            composition.SetServerRegistrar(new FrontEndReadOnlyServerRegistrar());
        }
        else
        {
            composition.SetServerRegistrar(new MasterServerRegistrar());
        }
    }
}

Registrars

public class FrontEndReadOnlyServerRegistrar : IServerRegistrar
{
    public IEnumerable<IServerAddress> Registrations
    {
        get { return Enumerable.Empty<IServerAddress>(); }
    }
    public ServerRole GetCurrentServerRole()
    {
        return ServerRole.Replica;
    }
    public string GetCurrentServerUmbracoApplicationUrl()
    {
        return null;
    }
}
public class MasterServerRegistrar : IServerRegistrar
{
    public IEnumerable<IServerAddress> Registrations
    {
        get { return Enumerable.Empty<IServerAddress>(); }
    }
    public ServerRole GetCurrentServerRole()
    {
        return ServerRole.Master;
    }
    public string GetCurrentServerUmbracoApplicationUrl()
    {
        return null;
    }
}

Lucene App Settings

<add key="WEBSITE_DISABLE_OVERLAPPED_RECYCLING" value="1" />
<add key="umbracoLocalTempStorage" value="EnvironmentTemp" />
<add key="Umbraco.Core.LocalTempStorage" value="EnvironmentTemp" />
<add key="Umbraco.Examine.LuceneDirectoryFactory" value="Examine.LuceneEngine.Directories.SyncTempEnvDirectoryFactory, Examine" />

Error Messages

Error 1

Cannot index queue items, the index is currently locked

Error 2

System.ApplicationException: Could not create an index searcher with the supplied lucene directory ---> Lucene.Net.Index.CorruptIndexException: doc counts differ for segment _x6: fieldsReader shows 40 but segmentInfo shows 41

Error 3

Lucene.Net.Store.LockObtainFailedException: Lock obtain timed out: SimpleFSLock@D:\home\site\wwwroot\App_Data\TEMP\ExamineIndexes\External\write.lock

Thanks in advance

Andy


Solution

  • The WEBSITE_DISABLE_OVERLAPPED_RECYCLING setting needs to be set in the Configuration page in your Azure portal. Having it in web.config is not going to help you.

    I think there is a lot of improvements in regards to locking issues in 8.4, so try to upgrade to at least 8.4.x.