Follow symlinks? about metro

[ericwooley]

Can we leave this open until it's resolved?

This is a huge PITA for code sharing, and developing modules. I'm not sure what your looking for in terms of a strong case, but it's definitely a major pain point for me, and module developers.

Currently I am trying to use lerna, and react-primitives to reorganize a project boilerplate to maximize code reuse, while maintaining upgradability by rn.

Lerna works by symlinking and installing your packages in a mono repo, which is an incredibly helpful pattern for code reuse, and completely broken when the react native packager won't follow symlinks.

[jeanlauliac]

I think we won't have time to deal with this anytime soon unfortunately. I'd love to reopen once we have a strong case.

[lacker]

symlinks not working in React Native is one of the biggest RN complaints, so if my understanding is correct and this would fix that, this would be great!

[cpojer]

Yeah, it should be open. We should find a fix but we are not actively invested in making this change ourselves right now. If the community can find a reasonable solution, we should merge it and make it work.

[haggholm]

I'd love to reopen once we have a strong case.

With npm 5, all local installs (npm i ../foo) are now symlinked. This makes it pretty rough to work with projects that have local dependencies. (That’s on top of the monorepo scenario.)

[aleclarson]

The OP was never clear about what exactly is broken with symlinks, but I'm going to assume he's only talking about registering symlinks with the file watcher (since that's what everyone is talking about).

The following solution:

• has a minimal footprint
• does not generate a config file
• automatically locates your locally developed packages

Steps

1. npm install get-dev-paths --only=dev

2. Add the following snippet to your metro.config.js module

const fs = require('fs')
const getDevPaths = require('get-dev-paths')

const projectRoot = __dirname
module.exports = {
  // Old way
  getProjectRoots: () => Array.from(new Set(
    getDevPaths(projectRoot).map($ => fs.realpathSync($))
  )),
  // New way
  watchFolders: Array.from(new Set(
    getDevPaths(projectRoot).map($ => fs.realpathSync($))
  ))
}

Questions

• Should we consider this issue closed now?
• Should metro adopt this technique to achieve a zero-config solution?
• Are there other issues with symlinks?

[GingerBear]

based on #1 (comment), I created a script that automated the workarounds with rn-cli-config.js to make npm link work https://gist.github.com/GingerBear/485f922a1e403739dc56d279925b216d

what it does is basically

• check in package.json dependencies and devDependencies, see if those are symlink in node_module
• if found, generate rn-cli-config.js with those symlink's real path
• react-native start with rn-cli-config

so the workflow is, do regular npm link first, then run node ./react-native-start-with-link.js to start bundler with symlink

[joearasin]

Punting on this issue is making a mess out of the ecosystem. By forcing dependencies on alternative builders for reasonably common setups, we end up stuck on brittle build stacks that are not tested as part of the official release pipeline. Upgrades then become dangerous.

[kschzt]

I'm in the lerna boat, too. The way I finally worked around this issue was to just explicitly add the packages in our monorepo to rn-cli.config.js, instead of having them as dependencies in the RN project's package.json (and symlinked into node_modules)

var config = {
  getProjectRoots() {
    return [
      path.resolve(__dirname), 
      path.resolve(__dirname, '../some-other-package-in-lerna-monorepo')
    ]
  }
}

May have changed for Metro. HTH

[gastonmorixe]

please fix this :(

[ktj]

Is the only solution at the moment to use haul or some custom shell scripts?

[sverrejoh]

@macrozone

I belive the problem is two-fold: One is that watchman, the tool in charge of monitoring files and triggering a compile whenever a file changes, doesn't work cross symlinks. So a file change within a symlinked folder will never trigger a re-compile.

The other problem is that React Native doesn't seem to follow symlinks outside the project root, but this you can fix by adding another project root through the react native config.

So first npm link like you did before, and then create a file called rn-cli-config.js, and put it in your root react native folder.

var path = require("path");
var config = {
    getProjectRoots() {
        return [
            // Keep your project directory.
            path.resolve(__dirname),

            // Include your forked package as a new root.
            path.resolve(__dirname, "../../../your-forked-package")
        ];
    }
}
module.exports = config;

Use the config together with your normal react-native command. Eg:
react-native start --config rn-cli.config.js

I believe this is what made it work for me (you just have to restart the bundler for it to recompile the linked package). Tell me if you have any issues, and good luck!

[MrLoh]

I made this script available via npm https://github.com/MrLoh/metro-with-symlinks.

Simply yarn add -D metro-with-symlinks and replace the start script in your package.json with node ./node_modules/metro-with-symlinks start. If anyone really needs support for nested symlinked packages, feel free to open a PR or create an issue in my repo.

[th317erd]

For everyone else having this problem, this works for me with the newest packager:

master-project/rn-cli.config.js

const path = require('path');
const fs = require('fs');

let blacklist,
    getPolyfills;

// Get blacklist factory
try {
  blacklist = require('metro-bundler/src/blacklist');
} catch(e) {
  blacklist = require('metro/src/blacklist');
}

// Get default react-native polyfills
try {
  getPolyfills = require('react-native/rn-get-polyfills');
} catch(e) {
  getPolyfills = () => [];
}

// See if project has custom polyfills, if so, include the path to them
try {
  let customPolyfills = require.resolve('./polyfills.js');
  getPolyfills = (function(originalGetPolyfills) {
    return () => originalGetPolyfills().concat(customPolyfills);
  })(getPolyfills);
} catch(e) {}

const moduleBlacklist = [
  //Add whatever you need to the blacklist for your project
  /node_modules[^\/]+\/.*/
];

const baseModulePath = path.resolve(__dirname, 'node_modules');

// NOTE: Scoped modules hasn't been fully tested yet. Please respond to
// let th317erd know if this code works with scoped modules
function getSymlinkedModules() {
  function checkModule(fileName, alternateRoots) {
    try {
      let fullFileName = path.join(baseModulePath, fileName),
          stats = fs.lstatSync(fullFileName);

      if (stats.isSymbolicLink()) {
        let realPath = fs.realpathSync(fullFileName);
        if (realPath.substring(0, (__dirname).length) !== __dirname)
          alternateRoots.push(realPath);
      }
    } catch (e) {}
  }

  function checkAllModules(modulePath, alternateRoots) {
    var stats = fs.lstatSync(modulePath);
    if (!stats.isDirectory())
      return alternateRoots;

    fs.readdirSync(modulePath).forEach((fileName) => {
      if (fileName.charAt(0) === '.')
        return;

      if (fileName.charAt(0) !== '@')
        checkModule(fileName, alternateRoots);
      else
        checkAllModules(path.join(modulePath, fileName), alternateRoots);
    });

    return alternateRoots;
  }

  return checkAllModules(baseModulePath, []);
}

function getExtraModulesForAlternateRoot(fullPath) {
  var packagePath = path.join(fullPath, 'package.json'),
      packageJSON = require(packagePath),
      alternateModules = [].concat(Object.keys(packageJSON.dependencies || {}), Object.keys(packageJSON.devDependencies || {}), Object.keys(packageJSON.peerDependencies || {})),
      extraModules = {};

  for (var i = 0, il = alternateModules.length; i < il; i++) {
    var moduleName = alternateModules[i];
    extraModules[moduleName] = path.join(baseModulePath, moduleName);
  }

  return extraModules;
}

//alternate roots (outside of project root)
var alternateRoots = getSymlinkedModules(),
//resolve external package dependencies by forcing them to look into path.join(__dirname, "node_modules")
    extraNodeModules = alternateRoots.reduce((obj, item) => {
      Object.assign(obj, getExtraModulesForAlternateRoot(item));
      return obj;
    }, {});

if (alternateRoots && alternateRoots.length)
  console.log('Found alternate project roots: ', alternateRoots);

module.exports = {
  getBlacklistRE: function() {
    return blacklist(moduleBlacklist);
  },
  getProjectRoots() {
    return [
      // Keep your project directory.
      path.resolve(__dirname)
    ].concat(alternateRoots);
  },
  extraNodeModules,
  getPolyfills
};

[sebirdman]

I've managed to link two react-native projects together with everything (at least react-native start and live reloading) working.

I have a library with common components, using typescript, that is npm linked to my main application

1. npm install on both items
2. npm link on YOUR_RN_LIBRARY
3. npm link YOUR_RN_LIBRARY on the main project
4. create rn-cli.config.js in the main project:

var path = require("path");
const metroBundler = require('metro-bundler');

var config = {
  extraNodeModules: {
    "react-native": path.resolve(__dirname, "node_modules/react-native"),
  },
  getProjectRoots() {
    return [
      // Keep your project directory.
      path.resolve(__dirname),
      path.resolve(__dirname, "LIBRARY_RELATIVE_PATH"),
    ];
  },
  getBlacklistRE: function() {
    return metroBundler.createBlacklist([
        /USER[/\\]PATH[/\\]TOLIBRARY[/\\]node_modules[/\\]react-native[/\\].*/
    ]);
  }
}
module.exports = config;

It seems like the blacklist must be the absolute path to your library on your file system. also for some reason i have to specify exactly where react-native is with extraNodeModules 😕

finally. npm link is working.

(note that i got this working on RN 0.47.2 - i think bundler changed createBlacklist in 48.X)

[MrLoh]

I've tried haul as well as an alternative, but it was still so unstable that it didn't make sense for me, your mileage may vary of course and maybe it's improved much since. However with a custom metro config everything has been working like a charm for me for a while.

The custom script I use to start the bundler is the following. It checks for symlinked packages and then generates a metro.config.js file based on that. It also checks the peer dependencies of those linked packages, making sure that those are available to the symlinked packages. Then the script starts the metro bundler, but you can also simply start the bundler yourself by passing the --config flag in other places, like for building for production, as the config file is written into your directory.

This script is heavily inspirend by the previous work of @GingerBear and @sebirdman in this thread.

/*
   - check symlink in depencency and devDepency
   - if found, generate rn-cli-config.js
   - react-native start with rn-cli-config

   Sources:
   https://github.com/facebook/metro/issues/1#issuecomment-346502388
   https://github.com/facebook/metro/issues/1#issuecomment-334546083
*/

const fs = require('fs');
const exec = require('child_process').execSync;

const getDependencyPath = (dependency) => fs.realpathSync(`node_modules/${dependency}`);

const getSymlinkedDependencies = () => {
  const packageJson = require(`${process.cwd()}/package.json`);
  const dependencies = [
  ...Object.keys(packageJson.dependencies),
  ...Object.keys(packageJson.devDependencies),
  ];
  return dependencies.filter((dependency) =>
  fs.lstatSync(`node_modules/${dependency}`).isSymbolicLink()
  );
};

const generateMetroConfig = (symlinkedDependencies) => {
  const symlinkedDependenciesPaths = symlinkedDependencies.map(getDependencyPath);

  const peerDependenciesOfSymlinkedDependencies = symlinkedDependenciesPaths
  .map((path) => require(`${path}/package.json`).peerDependencies)
  .map((peerDependencies) => (peerDependencies ? Object.keys(peerDependencies) : []))
  // flatten the array of arrays
  .reduce((flatDependencies, dependencies) => [...flatDependencies, ...dependencies], [])
  // filter to make array elements unique
  .filter((dependency, i, dependencies) => dependencies.indexOf(dependency) === i);

  fs.writeFileSync(
  'metro.config.js',
  `/* eslint-disable */
const path = require('path');
let blacklist
try {
  blacklist = require('metro-bundler/src/blacklist');
} catch(e) {
  blacklist = require('metro/src/blacklist');
}

module.exports = {
  extraNodeModules: {
  ${peerDependenciesOfSymlinkedDependencies
  .map((name) => `'${name}': path.resolve(__dirname, 'node_modules/${name}')`)
  .join(',\n\t\t')}
  },
  getBlacklistRE() {
  return blacklist([
  ${symlinkedDependenciesPaths
  .map(
  (path) =>
  `/${path.replace(/\//g, '[/\\\\]')}[/\\\\]node_modules[/\\\\]react-native[/\\\\].*/`
  )
  .join(',\n\t\t\t')}
  ]);
  },
  getProjectRoots() {
  return [
  // Include current package as project root
  path.resolve(__dirname),
  // Include symlinked packages as project roots
  ${symlinkedDependenciesPaths.map((path) => `path.resolve('${path}')`).join(',\n\t\t\t')}
  ];
  }
};`
  );
};

/* global process */

const symlinkedDependencies = getSymlinkedDependencies();
// eslint-disable-next-line no-console
console.log(`
Detected symlinked packaged:
${symlinkedDependencies
  .map((dependency) => `   ${dependency} -> ${getDependencyPath(dependency)}`)
  .join('\n')}
`);

generateMetroConfig(symlinkedDependencies, 'metro.config.js');
// eslint-disable-next-line no-console
console.log('Generated custom metro.config.js to support symlinks\n');

const command = process.argv[2];
const flags = process.argv.slice(3).join(' ');
exec(
  `node node_modules/react-native/local-cli/cli.js ${command} --config ../../../../metro.config.js ${flags}`,
  { stdio: [0, 1, 2] }
);

Maybe it would be an option for metro to do something like this internally, but someone from the core team would have to comment on that.

[joearasin]

@sverrejoh The specific situation I'm dealing with is that I have a monorepo set up (ideally using yarn workspaces) with a react-native app, a react-web app, and a shared library (that needs to be babelified). I'm trying to figure out how to make the shared library (+ dependencies) accessible to the react-native app.

[ericwooley]

@kschzt I will have to try that solution tonight.

I imagine that there will be more pressure on this issue once yarn workspaces are released. Until then, your solution may be the only way forward.

[sverrejoh]

@joearasin Not sure what your problem is, but it is possible to also add modules explicitly with the extraNodeModules setting:

var path = require("path");
var config = {
    extraNodeModules: {
        react: path.resolve(__dirname, "node_modules/react")
    }
}
module.exports = config;

[CrazyPython]

Improving the speed of Metro takes a backseat to this feature. Rewrite the bundler if you need to. Do something hacky. Do something duct tape-y. Because in the end, what matters is the library you ship.

[ericwooley]

I suspect it has to do with this line
according to the docs you may also need to check if it's a symlink. EG

//...
 this._moduleResolver = new ModuleResolver({
      dirExists: filePath => {
        try {
           var stats = fs.lstatSync(filePath);
          return  stats.isDirectory() || stats.isSymbolicLink();
        } catch (e) {}
        return false;
      },
// ...

Though I am short on time for digesting how to get started with this project at the moment. I may have more time over the weekend for a proper PR.

Or if someone else gets the chance to play around with it, that would be awesome.

I never have enough time to accomplish all the things I want to do :(

Edit: added "may", as the docs don't explicitly say you isDirectory() won't return true on a symlink, but I believe that is the case.

Edit^2: https://github.com/ericwooley/symlinkTest You do need to explicitly check on OSX Sierra at least, not sure about linux or windows.

Edit^3: the commit where the test was removed c1a8157

[haggholm]

I was unable to get things to work in something like a monorepo when trying @kschzt’s approach. If I have module A as a dependency and point rn-cli.config.js to it explicitly, then it will not be found by packages requiring it from inside the ‘root’ repo. In fact, the packager doesn’t even look! The structure is probably fairly obvious from a log excerpt:

Looking for JS files in
   /mnt/scratch/shared/petter/owl-devenv/client/root/local
   /mnt/scratch/shared/petter/owl-devenv/client/root/local/@client
   /mnt/scratch/shared/petter/owl-devenv/client/root/local/@client/util
   /mnt/scratch/shared/petter/owl-devenv/client/root 

React packager ready.

Loading dependency graph, done.
warning: the transform cache was reset.
error: bundling: UnableToResolveError: Unable to resolve module `@client/util/memoizeOnce` from `/mnt/scratch/shared/petter/owl-devenv/client/root/node_modules/@client/store/bind.js`: Module does not exist in the module map or in these directories:
  /mnt/scratch/shared/petter/owl-devenv/client/root/node_modules/@client/util

Why does it not check to see if the module exists inside the local/ directory? Beats me.

(react-native v0.45.0.)

[joonhocho]

@sverrejoh Thanks it worked for me.
I am using the following command instead in case your global react-native-cli does not match your project react-native version:

node node_modules/react-native/local-cli/cli.js start --config ../../../../rn-cli-config.js

[fatfatson]

need+1
now I use a watch-sync way to share common file:

https://www.npmjs.com/package/sync-files

[DylanVann]

@MrLoh Script worked great!

It would be nice if metro did something similar internally. As a temporary workaround you could publish that script as an npm module for people to use.

I think the problem @ak99372 is mentioning is that if you had local-package-a which depended on local-package-b then local-package-b would not be picked up by your script.

To fix that the script could instead just loop through all the folders in node_modules instead of looking at package.json.

[jeanlauliac]

Edit^3: the commit where the test was removed c1a8157

The test was removed because it was already broken for a long time, unfortunately (I forgot to put that in the changeset description apparently). Now I realised I shouldn't have removed it completely, only make verify the broken behavior.

I suspect it has to do with this line

dirExists is only used for debugging, not for resolution. The resolution is done based on the HasteFS instance provided by the jest-haste-map module. I believe, but may well be wrong, that a fix for symlinks will have to be implemented in jest-haste-map, both for its watchman and plain crawlers, as well for its watchman and plain watch modes. Since, last time I've heard (might have changed!), watchman doesn't handle symlink, we were hampered in these efforts.

[Alazoral]

We're trying to use yalc to test packages before we release them, but this deficiency means we have to publish, then test, which is pretty crazy.

[alexmngn]

I've been using the above and for regular stuffs it's been working pretty well.
Although I have issues when a dependency of my package is react-native.

As an example, I'm using Lerna to manage a RN project which looks like this:

/packages
  /ui
    /node_modules
    /src
      /Spinner.js
  /main
    /node_modules
    /src
      /index.js
    /rn-cli.config.js

I use import Spinner from 'ui'; in main/src/index.js. Spinner.js uses react-native-spinkit to display a loading indicator. react-native-spinkit tries to load react-native from /ui/node_modules, although because I've black-listed it to avoid the naming collision, it cannot find it. Is there any solution?

Unable to resolve react-native from react-native-spinkit/index.js [...]. Module does not exist in ui/node_modules

[dariocravero]

In case anyone is stuck with this and until the different issues are fixed, I made a little guide on how to use yarn workspaces with Create React App and Create React Native App (Expo) to share common code across. Hope you find it handy! https://learn.viewsdx.com/how-to-use-yarn-workspaces-with-create-react-app-and-create-react-native-app-expo-to-share-common-ea27bc4bad62 https://medium.com/viewsdx/how-to-use-yarn-workspaces-with-create-react-app-and-create-react-native-app-expo-to-share-common-ea27bc4bad62

@GingerBear, thanks for that snippet! I reworked it a bit so it picks up the list of workspaces and published it as a package here https://www.npmjs.com/package/metro-bundler-config-yarn-workspaces

[ivmos]

@fatfatson I use https://github.com/wix/wml which I think it's similar.
So is syncing file copies the only workaround by now? Thanks in advance

[pepijnmulders]

Got it to work with the following setup. Project structure is just an example, it shows at least the shared package out of the app and web dir.

Project structure:

• app
  • ...react-native project structure
  • react-native-start.js
• shared
  • dist
  • node_modules
  • src
    • the shared ES6 code which compiles to dist when watching or building
• web
  • ...reactjs project structure

Shared
create .babelrc with following content:
{ "presets": [ "stage-1" ], "plugins": [ [ "transform-runtime", { "polyfill": false, "regenerator": true } ] ] }

In case needed my package.json looks like this. Don't forget to change the name of the shared project. This is the endpoint npm link will use later on.

package.json ==> https://pastebin.com/raw/bLhYDUP5

Shared is now setup properly so run npm link:

cd ./shared
npm link

When developing just run npm run watch in de shared dir.

App
Create an alternative startup script so don't use react-native run-[platform] anymore. Create a startup script (EG: react-native-start.js) with the following content from pastebin.

react-native-start.js ==> https://pastebin.com/raw/jMv9vpq5

Now link the shared package and add the following line to the package.json:
cd ./app
npm link [package_name]_shared

add to package.json:
"[package_shared]_shared": "file:../shared",

From now on you just can start and use symlinks with react-native. This script creates another script called rn-cli.config.js so metro bundler knows what to blacklist and what to use.

node react-native-start.js

Web
No additional config is needed for web. Just link the npm package and add it to the package.json

link the shared package and add the following line to the package.json:
cd ./web
npm link [package_name]_shared

add to package.json:
"[package_shared]_shared": "file:../shared",

In case this still doesn't work let me know and i will see if i can help you out.

[marioharper]

Thank you @MrLoh I've found this script to be the least intrusive way of getting symlinks to work.

However, instead of replacing my start script with metro-with-symlinks start I've instead called metro-with-symlinks in my projects custom bootstrap command (generating the metro.config.js) and added a rn-cli.config.js that just exports metro.config.js. This has the benefit of not needing to update Xcode and Gradle to point to the config since react native automatically looks for rn-cli.config.js

[th317erd]

Really good question @MrLoh! For the transformers, polyfills, blacklists, and other stuff needed for my project (such as an universally incredibly legendarily complex and difficult problem to solve--that only the God's have the recipe to solve--such as resolving symlinks ;) ) in rn-cli.config.js I had to deep dive into their source code to figure it out. It would be absolutely fantastic if the teams responsible would better document this stuff.

[digitaldeus]

@th317erd Just wanted to mention that I was able to get your script working with Node 6, but looks like it fails with Node 8. Not sure what the difference is between the two. Looks like it's failing to find react for the linked module in the hastemap, even though react is listed in the extraNodeModules properly.

Disregard that first bit, just figured out the problem was that I put the rn-cli config into a bin/ directory and it needed to be at the root.

I also had to make a small and ugly modification to get this working with scoped packages. I modified the getSymlinkedModules function like so:

function getSymlinkedModules() {
  var alternateRoots = [];

  for (let i = 0, il = baseModules.length; i < il; i++) {
    let file = baseModules[i];
    if (file.charAt(0) !== '.') {
      if (file.charAt(0) === '@') {
        const scopedModules = fs.readdirSync(path.join(baseModulePath, file))
        scopedModules.forEach(innerFile => {
          if (innerFile.charAt(0) !== '.') {
            let fullInnerFile = path.join(baseModulePath, file, innerFile),
              innerStats = fs.lstatSync(fullInnerFile);

            if (innerStats.isSymbolicLink()) {
              let realPath = fs.realpathSync(fullInnerFile);
              if (realPath.substring(0, (__dirname).length) !== __dirname)
                alternateRoots.push(realPath);
            }
          }
        })
      }
      else {
        let fullFile = path.join(baseModulePath, file),
          stats = fs.lstatSync(fullFile);

        if (stats.isSymbolicLink()) {
          let realPath = fs.realpathSync(fullFile);
          if (realPath.substring(0, (__dirname).length) !== __dirname)
            alternateRoots.push(realPath);
        }
      }
    }
  }

  return alternateRoots;
}

Quick and dirty but it works

[MrLoh]

I updatet the metro-with-symlinks pacakge to create a rn-cli.config.js file that is used automatically now and updated the readme.

[mgcrea]

A simpler working rn-cli.config.js (but requires manual definition of linked modules):

const path = require('path');
const blacklist = require('metro/src/blacklist');

const LINKED_LIBS = [path.resolve(process.env.HOME, 'Developer/react-native/react-native-elements')];

module.exports = {
  extraNodeModules: {
    'react-native': path.resolve(__dirname, 'node_modules/react-native')
  },
  getProjectRoots() {
    return [
      path.resolve(__dirname),
      ...LINKED_LIBS
    ];
  },
  getBlacklistRE: function() {
    return blacklist(LINKED_LIBS.map(lib => new RegExp(`${lib}/node_modules/react-native/.*`)));
  }
};

[joearasin]

@sverrejoh This almost works for me. I'm using yarn workspaces, my config looks like:

const config = {
  getProjectRoots() {
    return [path.resolve(__dirname), path.resolve(__dirname, '..')];
  },
};

react-native start runs fine. Attempting to run react-native link breaks down with "Cannot find module". Is there some way I can set the module search paths?

[dariocravero]

@GingerBear that's gold! Thanks so much for posting that script. I just used it on crna/expo to use external modules too. I'll do a write up before the end of the week on how to go about it there. expo/create-react-native-app/issues/232

[adamconservis]

@evrimfeyyaz Yeah, I've tried putting the module in there, it's dependencies in there, and both at the same time, and I still get the same error.

[th317erd]

@DEUSD Thank you for the reply for scoped modules! @DEUSD @ProLoser @cpjolicoeur I updated my code above with the refined work of @DEUSD (thank you!). If one of you could test for me and let me know if it works for you (I don't currently use scoped modules) that would be fantastic. If it works I will add it to a gist.

[sfrdmn]

BTW scoped packages don't work in extraNodeModules currently, until #173 is merged

[aleclarson]

@mgcrea Please open a new issue for that specifically, so people can more easily find the solution.

Anyone else who finds a problem with symlinks should also open a new issue, instead of posting it here. I'm in favor of locking this mega-thread so the individual problems with symlinks can be addressed separately.

[Titozzz]

@mgcrea Actually I also need the blacklist cause in one workspace a have multiple app so i'm using this to exclude any other react native than mine (I'm using no-hoist on react-native-* packages to allow multiple versions to coexist):

blacklistRE: /(nameOfRootDirectory|packages[/\\](?!nameOfCurrentPackage).*)[/\\]node_modules[/\\]react-native[/\\]/,

(Replace nameOfRootDirectory and nameOfCurrentPackage)

@jwaldrip Please DM me on twitter so we don't spam here if you need further help

@aleclarson I agree, this thread should be locked and closed, and any new issue should be treated separately with an repro example and a use case.

[ericwooley]

dirExists is only used for debugging, not for resolution. The resolution is done based on the HasteFS instance provided by the jest-haste-map module. I believe, but may well be wrong, that a fix for symlinks will have to be implemented in jest-haste-map, both for its watchman and plain crawlers, as well for its watchman and plain watch modes. Since, last time I've heard (might have changed!), watchman doesn't handle symlink, we were hampered in these efforts.

Interesting.

I am pretty confused by this project, as I am not sure how it is imported by RN. But I manually edited this change into node_modules/react-native/packager/src/node-haste/dependencyGraph.js to use my suggested code and no longer got errors about symlinked modules not being found. I got a ton of other errors, i think related to multiple installs of react-native, but I seemed to have gotten much further in the process.

I also put a console log there to offer some insight, and it was definitely logging. So that line appears to be used by the react-native project while doing real work (as opposed to debugging).

After much frustration with this issue, it ended up being easier to PR the libs i need to work with haul, rather than try to tackle this problem. I wish I had more time for this issue, but it appears to be a pretty rough one to solve, given what @jeanlauliac said.

Hopefully this gets officially resolved soon. as I would much prefer to use the built in solution.

On a side note, haul seems to have fixed all the symlink issues. Is there a downside to haul that you know of? It seems like it could be something that could be officially brought into the fold.

Something that half-works is not symbolic linking but hard linking. You can't hard link directories, but you can use cp to copy all the directories and hard link every file rather than copy it with -al, e.g. cp -al <sourceDirectory> <destinationDirectory>.

The only catch is that it seems in order to trigger rebundling I have to save the changed file (even though the action of saving isn't changing the contents of the file).

@ericwooley Can you explain how your PR, haul, and storybooks (?) relate to the react-native symbolic link issue?

[ericwooley]

@bleighb

Haul is an alternative to the react-native packager that doesn't have issues with symlinks. They reference this issue because those PR's and issues are about issues with symlinking and the metro bundler.

As to your cp -r issue, you might have better luck with rsync. Which is something I considered at one point.

[ericwooley]

@kschzt @haggholm

How are you importing your files

1. import whatever from '../../otherPackage/someModules'?
2. import whatever from 'otherPackage/someModules'?
3. import whatever from 'someModules'?

I'm not really sure how roots work in this context, but the way your importing may be a factor. Or are you using the @providesModule syntax?

[haggholm]

@ericwooley 2(ish): import whatever from '@foo/bar/baz, in this case, with baz having .js, .android.js, and .ios.js versions for different platforms.

Both the project and I are new to React Native (migrating an existing web app); I’m not familiar with the @providesModule syntax. Perhaps I should look into that and see if it works…

[pocesar]

funny, it used to work (import { something } from 'symlinked-package'), now it doesn't anymore after upgrading react native from 0.43 to 0.45, anything changed in the packager from there?

[AshCoolman]

This lerna project seems to work with Symlinks too (react-native 0.40)

https://github.com/samcorcos/learna-react-native

[ericwooley]

I have played with that too @AshCoolman, and it does seem to work, but setting it up in a similar way with other versions has not worked.

[AshCoolman]

@ericwooley My experience too

[bakso]

In China, 90% local project development use cnpm which is build with npminstall, they are using symlinks to save disk volume. We think not supporting symlinks is not a good idea.

[joearasin]

^ Looks like there's progress being made over in jest on fixing the underlying issue by exposing a parameter to allow passing on watchman and following symlinks. If/when that's resolved, it may be possible to parameterize metro-bundler with something that passes through.

[jeanlauliac]

I would be very happy if someone would like to take over this task and takes the time to investigate and implement the necessary fixes, both in dependencies and in metro-bundler. Let's make sure efforts are not duplicated. We will not have bandwidth to implement this ourselves in the mid future I'm afraid, but I welcome PRs for review.

[Titozzz]

I can't use react-native 0.48 because I have to use haul to fix this issue and haul is not working properly with 0.48 right now... 😢

[macrozone]

I forked a npm package and want to work on it locally by using yarn link. But it's not possible with react-native because of this issue. Is there a workaround for this?

[macrozone]

thanks @sverrejoh, will try that out

[sebirdman]

@alexmngn do you have "extraNodeModules" specified to your main project? (like i have above)

If you're running react-native start from the main folder, it should be exactly the same path that i have.

[alexmngn]

@sebirdman I had it for "react" but not for "react-native". Having them both does the job for my config 👍 Thanks

[achuvm]

@Alazoral, another way you can test release packages is to do a npm pack which generates the .tgz file which is what is uploaded to npm. Then you can npm install ./path/to/modules.tgz and that should work just fine.

If you update, then you have to npm pack and npm install module.tgz again though.

[nicolascouvrat]

@GingerBear
Nice one!! Just made a slight modification as I wanted it to launch expo using the packager config you created through your script rather than just the packager:
https://gist.github.com/nicolascouvrat/68c546860f8c7b8eaeabda18a9baed6b

[adamconservis]

@GingerBear I'm having trouble getting this method to pick up dependencies of the sym linked dependency. Any suggestions or is this not possible?

[GingerBear]

@adamconservis what's the trouble you had? Originally I was testing with [email protected], and I just created an empty project with [email protected], both are working fine.

[adamconservis]

I have a React Native project that is working fine, and I would like to be able to develop my own modules that can be reused between this and our React front end. To test this, I took a dependency from the project and cloned it to my machine. Then, I symlinked the dependency into the main project and ran this script, with one change. I had the react-native execute run-ios instead of start and what happens is the system finds the dependency that was symlinked, but it has dependencies that the project can't find. I've tried 3 different things. Running just as explained. Then I tried running yarn on the dependency so it would grab all of it's dependencies, that didn't work. Then I tried adding the necessary dependencies of the dependency into the main project, and that didn't work.

[GingerBear]

@adamconservis
try not using run-ios but use start, run-ios seems ignore the rn-cli-config that specified by ---config

[adamconservis]

@GingerBear using start doesn't run the app though, how do I trigger the build?

[adamconservis]

@GingerBear If I run-ios with the packager that was started by the script, I still run into the same issue. require can't find the child of the dependency.

[GingerBear]

@adamconservis you can try run react-native run-ios to run the app, then exit the packager that started by run-ios, then run the script (which start the packager again with --config ../../../../rn-cli-config-with-links.js), then ues cmd+r to reload the app.

[adamconservis]

@GingerBear Unfortunately, this still results in the same issue.

[evrimfeyyaz]

@adamconservis Did you try adding those dependencies under extraModules in your rn-cli.config.js file?

[Titozzz]

Hello, I spent too much time trying to find workarounds for this issue, I finally got the start command to work, but the bundle one still fails to resolve symlinks.

Can someone create a tasks list with all the fixes that still need to be implemented in order to get symlinks to work ? I want to fix this, but right now I have no idea where to start.

Thanks in advance !

[MrLoh]

@adamconservis did you ever find a solution, I am having a similar issues with the peerDependencies of my package

[MrLoh]

@adamconservis I figured it out, you just need to pass any peerDependencies to extraNodeModules, as said here: #1 (comment)

[afilp]

I find that react-native link does not work with the extraNodeModules feature.

[MrLoh]

No you need to update the paths that react-native link writes manually to wear the packages actually are. There’s no workaround for that but it’s also quite easy to fix.

_{Sent with GitHawk}

[MrLoh]

No you just need to setup the custom metro config as mentioned before and if you have native modules in different folders, just adjust the paths to those modules in the ios and android code. Since this doesn't need to be bundled, it's super straight forward to use native modules in different folders. react-native link simply creates the paths for you and is a tiny convenience, just adjust the paths and you are good to go. iOS doesn't support symlinks so that could probably never be fixed.

[pepijnmulders]

I've setup everything properly and symlinks are working great using react-native: 0.53.0 in development so first of all thanks @GingerBear for creating an easy startup method.

I managed to link the assets as well but now facing another problem in Android.
The problem start when trying to build an release build in Xcode. First it returns an error main.jsbundle can't be found which is solved with the following command which builds the main.jsbundle manualy

react-native bundle --entry-file ./index.js --platform ios --dev false --bundle-output ios/main.jsbundle --config /absolute/path/to/rn-cli-config-with-links.js

After relinking the main.jsbundle in Xcode the error is gone and build fine but without any assets so i don't have any images or other static files in my release app.

./gradlew assembleRelease returns The name react-native was looked up in the Haste module map

Anyone facing the same issue or have a solution for this specific issue?

[chaseholland]

Incase it helps anyone else, I spent a lot of time trying to workaround this problem and ended up switching to Haul instead of Metro. Haul works with symlinks https://github.com/callstack/haul

[ak99372]

@MrLoh your script relies on linked package to be referenced in package.json

1. npm link does not place linked package to package.json
2. placing linked package into .json manually will break the npm install

Could you shine some light on linked package pre-requirements your script uses in regards to 1. and 2.? Thx

[MrLoh]

I'm using yarn, so not sure wheter that makes a difference but my linked packages are declared as bla-package: ../black-package and everything works fine.

[aiwbend]

since yarn does not create a symlink, unlike npm

[MrLoh]

not sure what you are talking about, yarn link certainly creates symlinks. But yes, you have to execute the linking, you can't simply declar it in your package.json, unless you are using workspaces.
https://yarnpkg.com/lang/en/docs/cli/link/

[MrLoh]

@marioharper really, RN automatically checks for a rn-cli.config.js file in the project root. Why aren’t things like that documented anywhere and for what obscure reason doesn’t it check for metro.config.js, which is the documented way to call the config file. I’ll test this and potentially update the name of the config file in our script.

[cpjolicoeur]

@th317erd Thanks for the script. It needed some small tweaks to work with org-namespaced modules, but was a big help.

[th317erd]

@cpjolicoeur No problem! If your changes can be contributed to help other people pass them off to me and I will update my post

[c-goettert]

for me @th317erd's solution did not work, still getting the duplicate module definition error when I try to start my react native app. @MrLoh's solution worked for me, if I rename the created metro.config.js to rn-cli.config.js. Unfortunatley, the commant react-native run-ios then only starts the packager, the app seems not to be started automatically anymore. Does anyone have a clue on how I could make this work again?
EDIT: It does work, my issue doesn't seem to be related to rn-cli.config.js

[ProLoser]

@cpjolicoeur what modifications did you make for org packages? @th317erd maybe we can put your script on a gist somewhere?

[th317erd]

@ProLoser I wouldn't mind at all. I would like to hear what modifications @cpjolicoeur made before we do though.

[th317erd]

@DEUSD @ProLoser @cpjolicoeur I realized this morning that I was referencing getPolyfills in the above script but didn't define it (in this public posting of the script). I have now updated the script to include custom polyfills (if the specified file ./polyfills.js exists in your project).

[cpjolicoeur]

Sorry, just catching up on this now. Been out for a while. I'll try to clean up my script to remove our hard-coded/personalized features and post what I used for orgs as well.

[digitaldeus]

@th317erd just tried your updated script, it's not including the scoped modules for me in that version. Also has to remove the line that included ./transformer to get it to work not sure if that was required. Glad the snippet was useful!

[th317erd]

Thanks @DEUSD ! Let me know if / how you fix it so I can update the script. Yeah... the transformer isn't supposed to be in there...

[ssavitzky]

This is very quick and dirty, but it works. I'm very new to the javascript toolchain, but I've been slinging makefiles for a long time. Goes into web/src/Makefile and app/src/Makefile; assumes that web, lib, and app are siblings.

### Makefile to get around lack of symlink support

# Rule to make subdirectories
lib/%/: ../../lib/%/
	mkdir -p $@
# Rule to copy source files from the shared library
lib/%.js: ../../lib/%.js
	cp -a $< $@

DIRS = $(addsuffix /,$(shell cd ../..; find lib -type d -print))
LIBS = $(shell cd ../..; find lib -name '*.js' -print | grep -v '[\#~]')

all:: $(DIRS)
all:: $(LIBS) | $(DIRS) # require the subdirs before copying files

.PHONY: build
build: $(LIBS) | $(DIRS)
	npm run build

You have to run make whenever you make a change in the library. If you keep forgetting, just run

while :; do sleep 10; make; done &

[Titozzz]

i'm not really sure what is broken right now cause i'm using metro with yarn workspaces without any issue. I just have a simple custom config

[jwaldrip]

@Titozzz Can you share your config? Would love to use metro again. We currently use haul and it's considerably slower.

[mgcrea]

Yarn workspaces are working fine because every dependency is hoisted to the root of the workspace. The build issue that arises with symlinks is when you need to link an external dependency (eg. in my case react-native-elements where I had to patch things and test against my app). So the workflow is:

1. Cloning the external module locally (react-native-elements)
2. npm install; npm link
3. Linking it in your project npm link react-native-elements

The issue arise because there is conflicting multiple react-native* dependencies (the one in your project & the one in the external lib, eg. react-native-elements).

However these build issues are to be expected and do happen with webpack as well (multiple copies of react errors, etc.)

It is fixable in a not really user-friendly way with getBlacklistRE.

With webpack, it is a bit more simple with the resolve option:

config.resolve.alias['react-native-elements'] = path.join(modulesPath, 'react-native-elements');

I think the two actionable things could be:

• Improve the docs regarding symlink usage (getBlacklistRE).
• Add a resolve API like webpack.

[jaridmargolin]

@aleclarson @Titozzz Do you mind explaining why you think this thread should be closed? As far as I can see, metro still does not support symlinks.

As @mgcrea state:

Yarn workspaces are working fine because every dependency is hoisted to the root of the workspace.

This thread is specifically about symlink support. Lerna / Yarn may individually work, but that does not mean that symlinks are working.

--

@aleclarson, I'm also not sure I understand the rationale here:

Anyone else who finds a problem with symlinks should also open a new issue, instead of posting it here.

Wouldn't it be harder to track the status of symlink support if it were spread amongst multiple issues?

--

Should metro adopt this technique to achieve a zero-config solution?

@aleclarson, if you think you solution should be built into metro, you could create PR? If metro maintainers accept, than perhaps this issue could be closed.... Until then, I personally think this thread should remain open. At the moment, It is the only method I have for being notified when/if progress is made.

[Titozzz]

Well I'm using yarn workspace with local packages that are symlinked to the root and it's working fine, so I'd like to find an example where it's not working. Why I think this thread could be closed is because it's a mix of lerna / yarn / symlinks with many different solutions to different problems, so I'm not sure if it's still relevant.