tfsnpmcontinuous-integrationnugetautomated-deploy

Path size issues related to working with node/npm on a build server


I have a build server that builds and deploys to our continuous integration (CI) environment and auto-deploys to Development, Staging, Live - all via TFS Build Definitions.

On the build server I need to compile Dust templates (before the deploy step) via the utility (dustc) downloaded with Dust. Locally when I run within Visual Studio, Dust is downloaded when Visual Studio (VS) is started into the folder node_modules, however this folder is generally not checked in (otherwise the numerous client-side libraries with versions quickly result in a management overhead)

Dust is downloaded via npm (I'm using v3.5.2). As I understand it, the standard way to download modules that use npm to download is as follows:-

The end result 'should' be Dust is downloaded into the project structure (within the node_modules folder) and I can then issue the command to compile the Dust templates

However the problem is when npm is downloaded via NuGet, the npm folder structure is massively nested and hence blows out the Windows path 260 character limit (https://github.com/nodejs/node-v0.x-archive/issues/6960) - consequently the build fails even before the job has had a chance to run npm to download Dust (note: I have reduced the length of the TFS folders however npm leaves very little room for any division of build names, projects, etc - eg. .../packages/Npm.3.5.2/node_modules/npm/node_modules/npm-install-checks/node_modules/npmlog/node_modules/are-we-there-yet/node_modules/readable-stream/node_modules/core-util-is/lib is around 170 characters)

I've read Node npm windows file paths are too long to install packages which suggests using verion 3.x, or using of npm-flatten/dedupe - but I'm still having issues - mostly because it is the NuGet step that fails - before it gets to being able to do anything with npm

A resolution could be to pick only files that are needed (i.e. from npm or probably easier [but less flexible] would be just the Dust files [dustc, etc]) and include in source control and not include npm in NuGet. But this is messy - if the files I checkin are updated (as they often are) I have to make sure all is still intact and runs

Is there a cleaner way ?


Solution

  • Even though the path length limitation is really annoying, the most effective and easy way still is spending some time tweaking your file/folder structure to make this work.

    For example: instead of \xx\Build\Drop\ProjectName, just use \xx\Build\Drop (or \xx\Builds) since the project name is also in the build name.

    For the long path issue in TFS, there had been a related uservoice and now completed.

    Fix 260 character file name length limitation

    We’ve removed the limitation from the BCL for the basic file manipulation functionality (CRUD). You can find more details here:

    https://blogs.msdn.microsoft.com/dotnet/2016/08/02/announcing-net-framework-4-6-2/

    Immo Landwerth Program Manager .NET