[Discuss]Problems about unstable numeric module ID. about metro

[mc-zone]

Thanks for your reply! @davidaurelio

Yes, I probably knew we should retain numeric module ID for Random Access Bundle because of it's special index table. Now your reply also help me realize more about it.

How about when using normal bundle, except unbundle, provide an option to let users choose, or define their own moduleIdFactory?

I can realize about bundle size increase that your've mentioned. But I think it maybe not so much.

In fact, our team are using a local fork of react-native based on 0.35, changed the createModuleIdFactory to use hash module ID. code like:

function createModuleIdFactory({projectRoots, prefix}) {
  const fileToIdMap = Object.create(null);
  const usedIds = {};
  return ({path: modulePath, name}) => {
    var relativePath = getPathRelativeToRoot(projectRoots, modulePath);
    if (!(modulePath in fileToIdMap)) {
      fileToIdMap[modulePath] = getModuleHashedPathId(relativePath, usedIds);
    }
    return fileToIdMap[modulePath];
  };
}
function getModuleHashedPathId(path, usedIds){
  var len = 4;
  var hash = crypto.createHash("md5");
  hash.update(path);
  var id = hash.digest("base64");
  while(usedIds[id.substr(0, len)]){
    len++;
  }
  id = id.substr(0, len);
  usedIds[id] = path;
  return id;
}

Part of the output bundle like this:

Now I've make a compare at one of our RN apps, which total 539 modules in production. Here is the size change betwteen numeric module ID and hash module ID (both with uglify, dev=false):

838KB - 847KB. only 10KB increased.

So in contrast, I prefer to use hash module ID for normal text bundle, it give us ability to split bundle by stable module ID.

What you can do as of today is to implement the postProcessModulesForBuck hook to order modules by path.

I haven't tried postProcessModulesForBuck yet. I will have a look at it. But I think maybe reorder modules by path can't keep stable when deps was added/removed, their are still numbers related.

If I have any mistake please correct me. Thanks again!

[rafeca]

Hey @mc-zone, @lishoulong : We recently merged f347e4f, which allows to customize the createModuleIdFactory() function, so if you use v0.24.1 you're going to be able to use your custom logic for generating the module ids.

[davidaurelio]

the new ModuleGraph implementation only needs module IDs at the very end of the build process.

We could count the modules in each chunk, and assign them ranges, e.g. 0–1000 for the main bundle, 1000–2000 for the first chunk, depending on the number of files in each chunk.

By adding some headroom we could minimize changes to chunks that should normally be unaffected. It’s a 20%/80% solution, i.e. not perfect, but maybe good enough.

Could metro-bundler support string module ID?

numbers allow for some interesting optimizations, specifically memory-friendly look-up tables. We use that for Random Access Bundles (“unbundle”, yet to be documented :-( ) in native code (C++).

Unique hashes have to have a certain length to avoid collisions, and for large bundles that can make a non-trivial difference in size.

Everything is a tradeoff, we have to find the right one.

Using hashes in Random Access Bundles means we would have to change lookup structures from tables to hash maps, and that is undesirable, as it would incur a small TTI penalty.

Currently we are focused on runtime performance for mobile. Stable module IDs are desirable, too, but we don’t want the latter at the expense of the former.

We want to address the “multiple app” scenario in React Native with clever bundle splitting that allows sharing code between chunks, while retaining the positive properties of numeric module IDs.

What you can do as of today is to implement the postProcessModulesForBuck hook to order modules by path. This should have neutral performance for plain text bundles, but creates significant TTI regressions when using a Random Access Bundle.

[zimo888]

I agree with @mc-zone 's opinion, We have a B2B project that likes the expo, developers upload their react-native code to service, after that, service sends developer's code to our customer's app and combine it.

We have to split the bundle file to a base bundle and developer's business bundles, If we use numeric module ID, because the base module has only one,Two different developer's generate there module ID will be same.
Only use moduleName or hashName can do these.
I Sincerely hope the bundle commands of the packager will be more flexible and more powerful,Thank you very much.

[lishoulong]

@mc-zone ，I use your createModuleIdFactory for hashed ids, but when I run "react-native run build --dev false ...", it error the followings "Unexpected token: punc ()) in file "/Users/lifeifei/Documents/react-native/sourceCode/newRNBundle/index.js" at line 2:26".

I guess this is the cause of --dev false, maybe uglify.

I only change the code for the createModuleIdFactory as yours, Is I use right?

thanks!

[mc-zone]

@lishoulong I'm not sure, haven't seen this error. Could you provide a repo about your code?

[lishoulong]

Hi,
I fork the official metro-bundler, and I change the method of createModuleIdFactory as of you mention above.

The blow is my commits, please take your time have a look.

lishoulong@8c9f3bd

[lishoulong]

@mc-zone , I know what my question is , the below is the part code I copy from the builded bundle.

var AnimatedAddition = require(helpcenter-AnimatedAddition); // helpcenter-AnimatedAddition = ./nodes/AnimatedAddition;

As you can see the required helpcenter-AnimatedAddition is not quoted , so the program look helpcenter-AnimatedAddition as a variable.The question is some require content is also quoted.

But I do not know where the problem is?