nikaple / nest-typed-config Goto Github PK
View Code? Open in Web Editor NEWIntuitive, type-safe configuration module for Nest framework ✨
License: MIT License
Intuitive, type-safe configuration module for Nest framework ✨
License: MIT License
For an example:
I have env variable:-> DB_PORT=9002
.env.yml
----------------
db:
port: ${DB_PORT}
Will this replace ${DB_PORT}
with actual value 9002
Is there a way i can achieve this?
[ ] Regression
[ ] Bug report
[ ] Feature request
[ ] Documentation issue or request
[X] Support request
I am trying to load several quite complex yml files with nest-typed-config
For that reason, I am using the directoryLoader. The yml files are similiar to docker-compose.yml
files.
There are section names, that are dynamic and not known to me. The user can specify these names.
The files are looking like this
test1:
type: A
endPoint: endpoint1.com
port: 9000
anyProp: 'abc'
test2:
type: B
endPoint: endpoint2.com
port: 443
path: /path/to/anything/
Depending on a type
property, each section can have different properties.
But of course, same type means same properties.
My RootConfig
looks like
export class AConfig{
@Allow()
type: 'A'
@Allow()
@IsUrl()
endPoint: string
@Allow()
@IsNumber()
port: number
@Allow()
@IsNotEmpty()
anyProp: string
}
export class RootConfig {
public readonly endpoints!: {
[key: string]: AConfig | BConfig
}
};
I see, that the yml files are loaded, but the validation fails.
Is there a easy way to achieve that?
I get this exception:
- config undefined does not match the following rules:
- unknownValue: an unknown value was passed to the validate function, current config is `undefined`
at Function.validateWithClassValidator (/usr/src/app/node_modules/nest-typed-config/lib/typed-config.module.ts:135:13)
at Function.getDynamicModule (/usr/src/app/node_modules/nest-typed-config/lib/typed-config.module.ts:52:20)
at Function.forRoot (/usr/src/app/node_modules/nest-typed-config/lib/typed-config.module.ts:23:17)
[ ] Regression
[x] Bug report
[ ] Feature request
[ ] Documentation issue or request
[ ] Support request
Originally posted by @imkh in #148 (comment)
- For some reason, having a nested configuration in the RootConfig class with the
@ValidateNested()
decorator doesn't work. You must add the custom validate function shown in the README. It seems like someone else already ran into this problem in this issue. The README ofclass-validator
does mention thatnested object must be an instance of a class, otherwise @ValidateNested won't know what class is target of validation.
so that might be the reason?
As explained in your comment here, "there should be no difference with builtin validate function and custom validate function.".
@ValidateNested()
should work without the need to implement a custom validate function (especially since this decorator will be used almost always if you have nested config classes, otherwise it defeats the purpose of this feature).
Minimum reproduction repo here: https://github.com/imkh/nest-typed-config-issue-149
I wasn't able to reproduce the bug in the basic example of this repo:
$ git clone [email protected]:Nikaple/nest-typed-config.git
$ cd nest-typed-config.git
$ cd examples/basic
$ npm install @nestjs/core @nestjs/platform-express # The example wouldn't start without installing those 2 dependencies
$ npm run start # Go to http://127.0.0.1:<port>, nested config seem to work as expected
/
Nest version:
@nestjs/cli 8.2.1
@nestjs/core 8.3.1
(both latest versions at the time of writing)
For Tooling issues:
- Node version: 16.13.2
- Platform: Mac
Others:
[ ] Regression
[x] Bug report
[ ] Feature request
[ ] Documentation issue or request
[ ] Support request
src/app.module.ts:3:10 - error TS2305: Module '"nest-typed-config"' has no exported member 'TypedConfigModule'.
3 import { TypedConfigModule } from 'nest-typed-config';
~~~~~~~~~~~~~~~~~
src/config/index.ts:2:10 - error TS2305: Module '"nest-typed-config"' has no exported member 'TypedConfigModuleOptions'.
2 import { TypedConfigModuleOptions, fileLoader } from 'nest-typed-config';
~~~~~~~~~~~~~~~~~~~~~~~~
src/config/index.ts:2:36 - error TS2305: Module '"nest-typed-config"' has no exported member 'fileLoader'.
2 import { TypedConfigModuleOptions, fileLoader } from 'nest-typed-config';
Nest version: X.Y.Z
For Tooling issues:
- Node version: XX
- Platform:
Others:
[ ] Regression
[x] Bug report
[ ] Feature request
[ ] Documentation issue or request
[ ] Support request
If using nested configs, nest-typed-config will not initialize a nested config when no environment variable was set for this config, even though default values for this configuration are present.
Output when starting (for the concrete ExampleConfig, see the example below):
Nest can't resolve dependencies of the AppService (?). Please make sure that the argument ExampleConfig at index [0] is available in the AppModule context.
It looks like when no environment variable for a nested config is set, the nested config will not be initialized.
When I set an environment variable that is contained inside the nested config, everything works just fine.
The nested config should be available, even though no environment variables for this config were set.
config.ts:
import { Type } from 'class-transformer';
import { IsInt, Min, ValidateNested } from 'class-validator';
export class ExampleConfig {
@IsInt()
@Type(() => Number)
@Min(0)
public readonly helloWorld: number = 1337;
@IsInt()
@Type(() => Number)
@Min(0)
public readonly justANumber: number = 42;
}
export class RootConfig {
@ValidateNested()
@Type(() => ExampleConfig)
public readonly example!: ExampleConfig;
}
app.module.ts:
import { Global, Module } from '@nestjs/common';
import { dotenvLoader, TypedConfigModule } from 'nest-typed-config';
import { RootConfig } from './config';
import { AppService } from './app.service';
@Global()
@Module({
imports: [
TypedConfigModule.forRoot({
schema: RootConfig,
load: dotenvLoader({
separator: '_',
})
})
],
providers: [
AppService
]
})
export class AppModule {}
app.service.ts:
import { Injectable } from '@nestjs/common';
import { ExampleConfig } from '../config';
@Injectable()
export class AppService {
constructor(
private readonly exampleConfig: ExampleConfig
) {}
}
Nest version: 9.0.11
For Tooling issues:
- Node version: v18.9.0
- Platform: Mac
Others:
Package manager: Yarn
The app fails to start when bundling with webpack, even if you exclude class-validator
from bundling.
Webpack copies the https://github.com/Nikaple/nest-typed-config/blob/main/lib/utils/imports.util.ts directly into the bundle, except it tries to replace the require
call but can't resolve the dependency, so instead replaces it with an empty context, which causes the requireFromRootNodeModules
to throw an exception when starting.
/***/ }),
/* 1368 */
/***/ ((__unused_webpack_module, exports, __webpack_require__) => {
"use strict";
Object.defineProperty(exports, "__esModule", ({ value: true }));
exports.plainToClass = exports.validateSync = void 0;
const requireFromRootNodeModules = (moduleName) => {
const modulePath = __webpack_require__(1369).resolve(moduleName, { paths: ['../..', '.'] });
// eslint-disable-next-line @typescript-eslint/no-var-requires
return __webpack_require__(1369)(modulePath);
};
exports.validateSync = requireFromRootNodeModules('class-validator').validateSync;
exports.plainToClass = requireFromRootNodeModules('class-transformer').plainToClass;
//# sourceMappingURL=imports.util.js.map
/***/ }),
/* 1369 */
/***/ ((module) => {
function webpackEmptyContext(req) {
var e = new Error("Cannot find module '" + req + "'");
e.code = 'MODULE_NOT_FOUND';
throw e;
}
webpackEmptyContext.keys = () => ([]);
webpackEmptyContext.resolve = webpackEmptyContext;
webpackEmptyContext.id = 1369;
module.exports = webpackEmptyContext;
I created a basic patch that looks to fix the issue
diff --git a/dist/loader/dotenv-loader.js b/dist/loader/dotenv-loader.js
index 2bd4f53600790a2626f6469070c8c20f880ea6bf..142087986e787d65b4b1f5d269e435d0394c2398 100644
--- a/dist/loader/dotenv-loader.js
+++ b/dist/loader/dotenv-loader.js
@@ -40,7 +40,7 @@ const debug_util_1 = require("../utils/debug.util");
let dotenv;
let dotenvExpand;
const loadEnvFile = (options) => {
- dotenv = (0, load_package_util_1.loadPackage)('dotenv', 'dotenvLoader');
+ dotenv = require('dotenv');
const envFilePaths = Array.isArray(options.envFilePath)
? options.envFilePath
: [options.envFilePath || (0, path_1.resolve)(process.cwd(), '.env')];
diff --git a/dist/utils/imports.util.js b/dist/utils/imports.util.js
index 6ba2c6d4bca3e4ecc0003881494f2c21452c7a8b..cda6abecb57a8b2fb4e0e31222318a714a166ba9 100644
--- a/dist/utils/imports.util.js
+++ b/dist/utils/imports.util.js
@@ -6,6 +6,16 @@ const requireFromRootNodeModules = (moduleName) => {
// eslint-disable-next-line @typescript-eslint/no-var-requires
return require(modulePath);
};
-exports.validateSync = requireFromRootNodeModules('class-validator').validateSync;
-exports.plainToClass = requireFromRootNodeModules('class-transformer').plainToClass;
+
+try {
+ exports.validateSync = requireFromRootNodeModules('class-validator').validateSync;
+} catch {
+ exports.validateSync = require('class-validator').validateSync;
+}
+
+try {
+ exports.plainToClass = requireFromRootNodeModules('class-transformer').plainToClass;
+} catch {
+ exports.plainToClass = require('class-transformer').plainToClass;
+}
//# sourceMappingURL=imports.util.js.map
Besides the requireFromRootNodeModules
, the loadPackage
function also causes issues with webpacks resolution, while it's a convenient shorthand i'd suggest inlining it instead to avoid the issue.
try {
dotenv = require('dotenv');
} catch (e) {
console.error(MISSING_REQUIRED_DEPENDENCY('dotenv', 'dotenvLoader'));
process.exit(1);
}
You could still have some sort of utility function if you want, like
dotenv = loadPackage('dotenv', 'dotenvLoader', () => require('dotenv'));
const MISSING_REQUIRED_DEPENDENCY = (name: string, reason: string) =>
`The "${name}" package is missing. Please, make sure to install this library ($ npm install ${name}) to take advantage of ${reason}.`;
export function loadPackage(packageName: string, context: string, requireFn): any {
try {
return requireFn()
} catch (e) {
console.error(MISSING_REQUIRED_DEPENDENCY(packageName, context));
process.exit(1);
}
}
Support being bundled
Set externals: []
in your webpack config
Nest version: 10.3.1
Nest-Typed-Config version: 2.9.2
For Tooling issues:
- Node version: 20
- Platform: Windows
Others:
[ ] Regression
[X] Bug report
[ ] Feature request
[ ] Documentation issue or request
[ ] Support request
Default values seem to override loaders.
I would expect loaders to always override matching default values.
NOTE: This is my first time using this library, so maybe I'm missing something simple?
import {
TypedConfigModule,
dotenvLoader,
selectConfig,
} from 'nest-typed-config';
import { IsString, IsNumber } from 'class-validator';
import { mapKeys, camelCase } from 'lodash';
export class Config {
@IsString()
readonly nodeEnv: string = 'test';
@IsString()
readonly host: string = '127.0.0.1';
@IsNumber()
readonly port: number = 50051;
@IsString()
get url(): string {
return `http://${this.host}:${this.port}`;
}
@IsString()
get protoPath(): string {
return this.nodeEnv === 'production' ? 'proto' : '../../../libs/proto';
}
}
export const ConfigModule = TypedConfigModule.forRoot({
schema: Config,
load: [
async () => {
const rawEnv = dotenvLoader()();
const mapped = mapKeys(rawEnv, (value, key) => camelCase(key));
console.log(mapped); // nodeEnv = 'development'
return mapped;
},
],
});
export const appConfig = selectConfig(ConfigModule, Config);
console.log(appConfig); // nodeEnv = 'test' (wrong? shouldn't the loader have precedence?)
Trying to do a simple remapping of ENV variables
Nest version: 13.9.6
For Tooling issues:
- Node version: v16.14.2
- Platform: macOS
Others:
When there is a syntax error in an config file ( I tested .yaml ), using a single loader throws and error. An array of loaders does not.
Issue can be traced here.
The behaviour to be configurable or at least consistent.
Using config:
with:
an_error
inside: the file
Throws:
export const ConfigModule = TypedConfigModule.forRoot({
schema: RootConfig,
load: fileLoader()
});
Does not throw:
export const ConfigModule = TypedConfigModule.forRoot({
schema: RootConfig,
load: [fileLoader()]
});
Nest version: 16.17.0
Nest-Typed-Config version: 2.7.0
For Tooling issues:
- Node version: XX
- Platform:
Others:
Environment variable substitution defined in .env doesn't seem to be substituted into .env.yaml even when ignoreEnvironmentVariableSubstitution: false
for file loader. Config file is defined below
// .env
PORT=9000
LOADER=fileLoader
app:
somePort: ${PORT:-12345}
envLoader: ${LOADER}
The current output is
app:
somePort: 12345
envLoader: undefined
Expected output
app:
somePort: 9000
envLoader: fileLoader
https://gist.github.com/Nathan-Bernardo/bcb98b2215571223299209b9088489f2
Nest version: 8.0.0
Nest-Typed-Config version: 2.9.1
For Tooling issues:
- Node version: 16.18.2
- Platform: Ubuntu 18.04
[ ] Regression
[ ] Bug report
[ ] Feature request
[x ] Documentation issue or request
[ ] Support request
Nested Validation requires attribute that is not mentioned in the docs.
@ValidateNested()
Consider replacing @Allow() with @ValidateNested() in the examples where it makes sense.
I had to dig through the tests to find an example of this class being used.
I realize this is a class-validator
package feature, but it is likely that many will be using
both these packages for the first time.
Was confused why DatabaseConfig url was not being validated:
export class DatabaseConfig {
@IsString()
public readonly url!: string;
}
export class RootConfig {
@Type(() => DatabaseConfig)
@Allow()
public readonly database!: DatabaseConfig;
}
Save others from frustration of figuring out why validation only seems to run on the RootConfig and
not on any of its properties
Just switched Allow with ValidateNested here and it works:
export class DatabaseConfig {
@IsString()
public readonly url!: string;
}
export class RootConfig {
@Type(() => DatabaseConfig)
@ValidateNested()
public readonly database!: DatabaseConfig;
}
Doc only
[X] Regression
[ ] Bug report
[ ] Feature request
[ ] Documentation issue or request
[ ] Support request
Seems that expanding env variables is not working anymore in v2.5.1.
Setting the expandVariables: true
in module config for dotenvLoader
gives error
config = dotenvExpand({ parsed: config }).parsed!;
^
TypeError: dotenvExpand is not a function
Reverting to v2.4.7 helped solving the issue.
Expanding env variables works as described in documentation.
This should be enough to reproduce. Create a .env file with following contents:
foo=123
bar=${foo}456
Create a simple config file:
// config.ts
export class AppConfig {
foo: string;
bar: string;
}
In your app.module.ts
file configure the NestTypedConfig:
// app.module.ts
import { TypedConfigModule, dotenvLoader } from 'nest-typed-config';
import { AppConfig } from './config';
... // other imports from nestjs
@Module({
imports: [
TypedConfigModule.forRoot({
schema: AppConfig,
load: dotenvLoader({
expandVariables: true,
}),
})
],
})
export class AppModule {}
Start the project and the error will occur.
Nest version: 9.2.0
Nest-Typed-Config version: 2.5.1
For Tooling issues:
- Node version: 18.14.1
- Platform: Ubuntu
Others:
Environment keys aren't properly cased or camel-cased as is typical when writing the code.
It should be documented how to perform common key transforms to coerce and transform the key and value as needed for loading an app in dev mode using a yaml file and/or in production using environment variables.
Use multiple loaders per https://github.com/Nikaple/nest-typed-config#using-multiple-loaders
Note that if one has a config like:
export class DatabaseConfig {
@IsNumber()
public readonly port!: number;
@IsString()
public readonly userName: string;
}
Passing the right data in via a config file works easily, but getting the dotenv loader to properly deal with environment variables is not as well documented.
The easiest fix in the documentation is to note in a single spot how to handle what I believe is a relatively common pattern of local config files vs. production environment variable usage. Taking the above example, one needs to make a few tweaks to make that schema work for environment variables like so:
export class DatabaseConfig {
@IsNumber()
@Type(() => Number)
public readonly port!: number;
@IsString()
public readonly userName: string;
}
This is documented in the README under variable substitution, but I missed it the first few times trying to find the common handling for environment variables. The section here documenting transforming the variable manually also lends itself to confusing the ideal path:
https://github.com/Nikaple/nest-typed-config#transforming-the-raw-configuration
The last bit missing above is how to get an environment variable into the key format of userName
as shown above. I used a key transform like this to solve that generically in a way that'd work for all keys when nesting:
TypedConfigModule.forRoot({
schema: RootConfig,
load: [
fileLoader(),
dotenvLoader({
separator: '__',
keyTransformer: (key) =>
key
.toLowerCase()
.replace(/(?<!_)_([a-z])/g, (_, p1) => p1.toUpperCase()),
}),
],
}),
This way DATABASE__USER_NAME=testuser
is properly tranlated to the database config of userName
.
I think having an example of how to handle the key transform for common camel-cased property names along with the recommendation of using the @Type
based value transform would be beneficial to many.
Current 2.6.0 release requires NestJS < 10 as a peer dependency.
npm ERR! Could not resolve dependency:
npm ERR! peer @nestjs/common@">= 6.10.0 < 10" from [email protected]
npm ERR! node_modules/nest-typed-config
npm ERR! nest-typed-config@"^2.6.0" from the root project
npm ERR!
npm ERR! Conflicting peer dependency: @nestjs/[email protected]
npm ERR! node_modules/@nestjs/common
npm ERR! peer @nestjs/common@">= 6.10.0 < 10" from [email protected]
npm ERR! node_modules/nest-typed-config
npm ERR! nest-typed-config@"^2.6.0" from the root project
New Release with NestJS 10 support. Main already has it fixed with the following pull request:
This might be a general nestjs question, but I figured I would ask here as this is probably a use case others may have. Basically, I am developing an App -> Lib relationship with strict config checking (to prevent developers missing setting an environment variable).
The requirement here is that the App needs to fetch a bunch of variables from external source (AWS Secret Manager). This works fine and the Config is type checked after loading. However, I have a library that depends on some of the configs (a subset). I want there to be validation on the library's module as well. But I can't figure out how to get the library to only load after the App has loaded.. I'm getting undefined values for my config values within the library getting booted up. See illustration below:
AppModule
import { getSecret } from '@aws-lambda-powertools/parameters/secrets'
import { Config } from 'app/config'
/*
public readonly VAR_1!: string
public readonly VAR_2!: string
public readonly VAR_3!: string
public readonly VAR_4!: string
*/
@Module({
imports: [
TypedConfigModule.forRootAsync({
isGlobal: true,
schema: Config,
load: [
dotenvLoader(),
async () => {
const secret = await getSecret('my-secret', {
transform: 'json',
})
return secret
},
],
}),
ServiceModule.registerAsync({
inject: [Config],
}),
],
providers: [AppService],
})
export class AppModule {}
and then in my lib (ServiceModule):
import { HttpModule, HttpModuleAsyncOptions, HttpService } from '@nestjs/axios'
import { Config } from 'lib/service/config'
/*
public readonly VAR_3!: string
public readonly VAR_4!: string
*/
export const SERVICE_API_REST = 'SERVICE_API_REST'
@Module({})
export class ServiceModule {
static registerAsync(options: any): DynamicModule {
return {
module: CoreSVCSModule,
imports: [
HttpModule.registerAsync({
inject: [Config],
useFactory: async (config: Config) => {
return {
url: config.VAR_3,
headers: {
'Content-Type': 'application/json',
'api-key': config.VAR_4,
},
}
},
} as HttpModuleAsyncOptions),
],
providers: [
{
provide: SERVICE_API_REST,
inject: [HttpService],
useFactory: (client: HttpService) => client,
},
],
exports: [SERVICE_API_REST],
}
}
}
How can I change this to work as desired?
Illustration of developing libraries that share configs from a parent app, enforcing the same checks and informing developers which configs are needed from parent.
Nest version: 10
Nest-Typed-Config version: 2.6.0
For Tooling issues:
- Node version: 20
- Platform: Mac
Others:
nx monorepo
[ ] Regression
[x ] Bug report
[ ] Feature request
[ ] Documentation issue or request
[ ] Support request
Isuee : Default value is skipped in nested object when 1 or up property is define in env.
Nest version: 8.0.0
For Tooling issues:
- Node version: 14
- Platform: window
Others:
[x] Regression
[ ] Bug report
[ ] Feature request
[ ] Documentation issue or request
[ ] Support request
The release workflows kind of has false positive. The CI still passes because it's not really an error, but the release not triggered.
> semantic-release
[10:01:31 AM] [semantic-release] › ℹ Running semantic-release version 18.0.0
[10:01:31 AM] [semantic-release] › ✔ Loaded plugin "verifyConditions" from "@semantic-release/npm"
[10:01:31 AM] [semantic-release] › ✔ Loaded plugin "verifyConditions" from "@semantic-release/github"
[10:01:31 AM] [semantic-release] › ✔ Loaded plugin "verifyConditions" from "@semantic-release/git"
[10:01:31 AM] [semantic-release] › ✔ Loaded plugin "analyzeCommits" from "@semantic-release/commit-analyzer"
[10:01:31 AM] [semantic-release] › ✔ Loaded plugin "generateNotes" from "@semantic-release/release-notes-generator"
[10:01:31 AM] [semantic-release] › ✔ Loaded plugin "prepare" from "@semantic-release/npm"
[10:01:31 AM] [semantic-release] › ✔ Loaded plugin "prepare" from "@semantic-release/git"
[10:01:31 AM] [semantic-release] › ✔ Loaded plugin "publish" from "@semantic-release/npm"
[10:01:31 AM] [semantic-release] › ✔ Loaded plugin "publish" from "@semantic-release/github"
[10:01:31 AM] [semantic-release] › ✔ Loaded plugin "addChannel" from "@semantic-release/npm"
[10:01:31 AM] [semantic-release] › ✔ Loaded plugin "addChannel" from "@semantic-release/github"
[10:01:31 AM] [semantic-release] › ✔ Loaded plugin "success" from "@semantic-release/github"
[10:01:31 AM] [semantic-release] › ✔ Loaded plugin "fail" from "@semantic-release/github"
[10:01:35 AM] [semantic-release] › ℹ This test run was triggered on the branch main, while semantic-release is configured to only publish from next, alpha, therefore a new version won’t be published.
semantic-release
not configured to use main
branch by default, you need to configure the configs 😉
I have opinionated shareable config too if you don't mind. It's configured to use main
branch too alongside with next
, alpha
, beta
, and maintenance version branch (ex: 2.x
). It enforces the conventional commits specs and produced this kind of release notes and changelogs.
No CI error, and the package published to NPM successfully.
New commits not published automatically.
Installing nest-typed-config
installs the following optional dependencies:
While it is documented to use npm install --no-optional
, this is not supported in other package managers and is not the actual use case for optionalDependencies
.
If a dependency can be used, but you would like npm to proceed if it cannot be found or fails to install, then you may put it in the optionalDependencies object.
— https://docs.npmjs.com/cli/v9/configuring-npm/package-json?v=true#optionaldependencies
Similar to the dependencies field, except that these entries will not be required to build properly should they have any build script. Note that such dependencies must always be resolvable (otherwise we couldn't store it in the lockfile, which could lead to non-reproducible installs), but those which list os / cpu / libc fields will not be fetched unless they match the current system architecture.
This field is usually not what you're looking for, unless you depend on the fsevents package. If you need a package to be required only when a specific feature is used then use an optional peer dependency. Your users will have to satisfy it should they use the feature, but it won't cause the build errors to be silently swallowed when the feature is needed.
— https://yarnpkg.com/configuration/manifest#optionalDependencies
The dependencies should instead be peerDependencies
with peerDependenciesMeta
setting optional
to true
.
Ideally, unused dependencies are not included when installing a package.
Nest version: 9.0.11
Nest-Typed-Config version: 2.5.2
For Tooling issues:
- Node version: v18.16.0
- Platform: Linux
Others:
Package manager: [email protected] and [email protected]
[ ] Regression
[ ] Bug report
[ ] Feature request
[x] Documentation issue or request
[ ] Support request
Examples on how to use this package with multiple environments are missing.
An example to show how this package can be use to load the right .env
file
My use case:
.
├── src
│ └── ...
└── env
├── .env.development
├── .env.production
└── .env.staging
each .env
must follow the same EnvironmentVariables
interface. The one that will be loaded is defined by process.env.NODE_ENV
value. Also, I have an application config and database config that will use the env. vars.
interface EnvironmentVariables {
DB_PORT: number
APP_PORT: number
}
When I try to add the nest typed config as a dependency I get this error please verify that package.json has a valid main entry.
The issue I think mainly has to do with the class-validator dependency but I am not quite sure.
I can explain in more detail if needed
Error: Cannot find module '/home/almedin/Desktop/rekog/provatar/node_modules/.pnpm/@[email protected]_@[email protected]_@[email protected]_@nestjs+core@9_nhrtj2kto3prduyeup3bce5y4q/node_modules/graphql-subscriptions/dist/index.js'. Please verify that the package.json has a valid "main" entry
at tryPackage (node:internal/modules/cjs/loader:444:19)
at Function.Module._findPath (node:internal/modules/cjs/loader:715:18)
at Function.Module._resolveFilename (node:internal/modules/cjs/loader:1130:27)
at Function.Module._load (node:internal/modules/cjs/loader:985:27)
at Module.require (node:internal/modules/cjs/loader:1235:19)
at require (node:internal/modules/helpers:176:18)
at Object.<anonymous> (/home/almedin/Desktop/rekog/provatar/node_modules/.pnpm/@[email protected]_@[email protected]_@[email protected]_@nestjs+core@9_nhrtj2kto3prduyeup3bce5y4q/packages/query-graphql/src/subscription/index.ts:2:1)
at Module._compile (node:internal/modules/cjs/loader:1376:14)
at Object.Module._extensions..js (node:internal/modules/cjs/loader:1435:10)
at Module.load (node:internal/modules/cjs/loader:1207:32)
```
## Expected behavior
I should be able to use the class-validator and start the project with no errors even when im using class-validator
<!-- Describe what the desired behavior would be. -->
I should be able to start the project when i
## Minimal reproduction of the problem with instructions
<!-- Please share a repo, a gist, or step-by-step instructions. -->
## What is the motivation / use case for changing the behavior?
<!-- Describe the motivation or the concrete use case. -->
## Environment
<pre><code>
Nest version: X.Y.Z
Nest-Typed-Config version: X.Y.Z
<!-- Check whether this is still an issue in the most recent Nest version -->
For Tooling issues:
- Node version: XX <!-- run `node --version` -->
- Platform: <!-- Mac, Linux, Windows -->
Others:
<!-- Anything else relevant? Operating system version, IDE, package manager, ... -->
</code></pre>
[ ] Regression
[x] Bug report
[ ] Feature request
[ ] Documentation issue or request
[ ] Support request
Simple transformation in the normalize
function work as expected, but converting them to other datatypes (for eg Uint8Array) does not work.
The types and values should be preserved.
Stackblitz: https://stackblitz.com/edit/nestjs-starter-demo-uus4oz?file=src/config.ts
In this example, I convert TEST
into an Uint8Array
. When I print that inside the function itself, I get the expected output. But doing so in a service does not give the correct output.
All environments
[ ] Regression
[ ] Bug report
[ ] Feature request
[ ] Documentation issue or request
[x] Support request
npm ERR! code ERESOLVE
npm ERR! ERESOLVE could not resolve
npm ERR!
npm ERR! While resolving: [email protected]
npm ERR! Found: @nestjs/[email protected]
npm ERR! node_modules/@nestjs/common
npm ERR! @nestjs/common@"^9.0.5" from the root project
npm ERR! peer @nestjs/common@"^8.2.3 || ^9.0.0" from @nestjs/[email protected]
npm ERR! node_modules/@nestjs/apollo
npm ERR! @nestjs/apollo@"^10.0.19" from the root project
npm ERR! 9 more (@nestjs/axios, @nestjs/core, @nestjs/graphql, ...)
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peer @nestjs/common@">= 6.10.0 < 9" from [email protected]
npm ERR! node_modules/nest-typed-config
npm ERR! nest-typed-config@"^2.4.1" from the root project
We should expect the major version releases of nestjs to be included in the perrDependency tree
Nest version: 9.0.5
For Tooling issues:
- Node version: 16.15.1
- Platform: Mac
[ ] Regression
[ ] Bug report
[x] Feature request
[ ] Documentation issue or request
[ ] Support request
that introduces breaking changes
There are couple optional features that requires some 3rd-party package to work, and these packages are always loaded (due to the following barrel file)
nest-typed-config/lib/loader/index.ts
Lines 1 to 4 in 048f6e4
Only load the dependencies used by the loader(s) that will be used in the project.
So, instead of
import { TypedConfigModule, fileLoader } from 'nest-typed-config';
we must do this:
import { TypedConfigModule } from 'nest-typed-config';
import { fileLoader } from 'nest-typed-config/file-loader';
// or (both working in the same way)
import fileLoader from 'nest-typed-config/file-loader';
BONUS: only install dependencies that will be used in the end. How? moving hard-dependencies to optional dependencies instead.
nest-typed-config/package.json
Line 30 in 048f6e4
nest-typed-config/package.json
Line 31 in 048f6e4
nest-typed-config/package.json
Lines 37 to 38 in 048f6e4
nest-typed-config/package.json
Line 43 in 048f6e4
Let's say I want to use only the dotenv loader (or my own custom loader) in my Nodejs project, thus I don't need to load (and install) all other loaders and their dependencies.
Lazy-loading will decrease the bootstrap time and the bundle size of this package (if you choose to move few prod deps. to optional deps.).
This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.
These updates are currently rate-limited. Click on a checkbox below to force their creation now.
@nestjs/common
, @nestjs/core
, @nestjs/platform-express
, @nestjs/testing
)node
, @types/node
)These updates have all been created already. Click a checkbox below to force a retry/rebase of any.
debug
, @types/debug
)@typescript-eslint/eslint-plugin
, @typescript-eslint/parser
)These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.
.github/workflows/build.yml
actions/checkout v3
pnpm/action-setup v2
actions/setup-node v3
coverallsapp/github-action v2.2.0
.github/workflows/codeql.yml
actions/checkout v3
github/codeql-action v2
github/codeql-action v2
github/codeql-action v2
.github/workflows/doc.yml
actions/checkout v3
pnpm/action-setup v2
actions/setup-node v3
crazy-max/ghaction-github-pages v3.1.0
.github/workflows/release.yml
actions/checkout v3
pnpm/action-setup v2
actions/setup-node v3
coverallsapp/github-action v2.2.0
package.json
chalk 4.1.2
class-transformer 0.5.1
class-validator ^0.14.0
debug 4.3.4
lodash.frompairs 4.0.1
lodash.merge 4.6.2
set-value ^4.1.0
@commitlint/cli 17.7.1
@iarna/toml 2.2.5
@latipun7/commitlintrc 1.1.3
@latipun7/releaserc ^2.1.0
@nestjs/axios 3.0.0
@nestjs/cli 10.1.17
@nestjs/common 10.2.5
@nestjs/core 10.2.5
@nestjs/platform-express 10.2.5
@nestjs/testing 10.2.5
@types/debug 4.1.8
@types/express 4.17.17
@types/jest 29.5.5
@types/lodash.frompairs 4.0.7
@types/lodash.merge 4.6.7
@types/node 18.17.17
@types/set-value ^4.0.1
@typescript-eslint/eslint-plugin 5.62.0
@typescript-eslint/parser 5.62.0
axios 1.5.0
cosmiconfig 8.3.6
dotenv 16.3.1
dotenv-expand 10.0.0
eslint 7.32.0
eslint-config-prettier 9.0.0
eslint-plugin-import 2.28.1
eslint-plugin-prettier 4.2.1
husky 8.0.3
jest 29.7.0
lint-staged 14.0.1
parse-json 7.1.0
prettier 2.8.8
reflect-metadata 0.1.13
rimraf 5.0.1
rxjs 7.8.1
semantic-release ^22.0.0
ts-jest 29.1.1
typedoc 0.25.1
typescript 5.2.2
yaml 2.3.2
@iarna/toml >= 2.2.5
@nestjs/axios >= 0.1.0
cosmiconfig >= 8.0.0
dotenv >= 16.0.0
dotenv-expand >= 10.0.0
parse-json >= 5.2.0
yaml >= 1.10.2
@nestjs/common >= 6.10.0 < 11
reflect-metadata >= 0.1.12 < 0.3
rxjs >= 6.0.0 < 8
pnpm 7.33.1
rxjs 7.8.0
axios 0.27.2
.nvmrc
node v18
[ ] Regression
[x] Bug report
[ ] Feature request
[ ] Documentation issue or request
[ ] Support request
We use environment variables from .env
file for local development and set them in deployment settings in cloud.
Config properties are in camelCase, but ALL_CAPS is used for environment variables.
Some config options are optional and have default values.
To support this requirements we use dotenvLoader
to load .env
and normalize
function to map values.
export class Config {
@IsInt()
readonly propertyWithDefaultValue: number = 4321;
}
TypedConfigModule.forRoot({
schema: Config,
load: dotenvLoader(),
normalize(config) {
return {
propertyWithDefaultValue: config.UNDEFINED_ENV_PROPERTY,
};
},
}),
It ignores default value and applies undefined
.
Use default value if value is undefined
.
exposeDefaultValues: true
in plainToClass
method of class-transformer
makes it works.
See PR with fix and test for this case.
[ ] Regression
[x] Bug report
[x] Feature request
[ ] Documentation issue or request
[ ] Support request
Using the env loader using only environment variables, the variables not specified in the config file still get added to the config object.
To only have the environment variables in the config that are specified as class fields.
app.module.ts
:imports: [
TypedConfigModule.forRoot({
schema: Config,
load: dotenvLoader({
ignoreEnvFile: true,
ignoreEnvVars: false,
}),
}),
...
],
export default class Config {
@IsNumber()
public readonly FOO: number;
@IsString()
public readonly BAR: string;
}
FOO
, BAR
, and BAZ
.FOO
, BAR
, as well as BAZ
.During development, this behaviour clutters the config a lot. Most programmers have tons of environment variables on different machines, making it hard to keep track of what configs we actually want during code.
This also makes mistakes easy to miss, as one might try to use variable BAZ
during development, which would work just fine as it is put in the config object, without it actually being specified in the config file.
Nest version: 8.2.1
[ ] Regression
[ ] Bug report
[x] Feature request
[ ] Documentation issue or request
[ ] Support request
imports: [TypeOrmModule.forRoot()]
(without passing a configuration object to forRoot()
) is breaking even thought I specify configuration in environment variables as .env
file.
Specifying TypeORM configuration as Environment Variables as .env
file should pass and logged in to database with only imports: [TypeOrmModule.forRoot()]
(without passing a configuration object to forRoot()
).
TypeORM accept environment variables as configuration so it can't be used with dependency injection.
Nest version: 8.1.2
For Tooling issues:
- Node version: 16.13.0
- Platform:
Others:
I would like to use fileLoader
but NODE_ENV
is hardcoded in its behaviour now (https://github.com/Nikaple/nest-typed-config/blob/main/lib/loader/file-loader.ts#L66)
Add ability to use arbitrary ENV variable when loading config files. The default value for it is NODE_ENV. But overall NODE_ENV isn't designed for specifying deployment env rather the application modes (https://stackoverflow.com/a/53368078)
Nest version: X.Y.Z
Nest-Typed-Config version: X.Y.Z
For Tooling issues:
- Node version: XX
- Platform:
Others:
[ ] Regression
[ ] Bug report
[x] Feature request
[ ] Documentation issue or request
[ ] Support request
No documentation when importing from nest-typed-config
, removeComments
should be set to false
in tsconfig.json
The lodash.set package has an open CVE: https://security.snyk.io/vuln/SNYK-JS-LODASHSET-1320032. The maintainer of Lodash will not release a patched version, and he considers the per-method packages lodash/lodash#3793 (comment).
We should be able to use set-value as a drop-in replacement.
right now if you have such a config yaml file for example:
logs:
colorize: ${LOGS_COLORIZE}
and have a default value in the class
export default class LoggerConfig {
@IsBoolean()
@Allow()
colorize = true;
}
you will get an error when the app is starting
Error: Environment variable is not set for variable name: 'LOGS_COLORIZE'
It will be nice to have the ability to specify a default value in the file itself. Like this:
logs:
colorize: ${LOGS_COLORIZE:true}
it just will be convenient, I've seen this behavior in all config libs in java and .net.
[ ] Regression
[ ] Bug report
[x] Feature request
[ ] Documentation issue or request
[ ] Support request
Would it be possible to add a capability for a runtime check of the configuration? For example:
[x] Regression
[ ] Bug report
[ ] Feature request
[ ] Documentation issue or request
[ ] Support request
Installing v2.4.7 does include the dist
.
Installing v2.5.0 does not include the dist
.
The dist
folder to be included in the package, so the index.ts
can export everything from it.
npm init
npm install nest-typed-config
dist
folder is not present.The package is currently unusable.
Nest version: N/A
For Tooling issues:
- Node version: 16.16.0
- Platform: Windows
Others:
This happens using both yarn
and npm
.
[ ] Regression
[X] Bug report
[ ] Feature request
[ ] Documentation issue or request
[ ] Support request
If you use Environment Variable Substitution by setting ignoreEnvironmentVariableSubstitution
to false and the environment variable is not set it throws an error crashing the app.
Info Webpack is building your sources...
webpack 5.73.0 compiled successfully in 416 ms
Type-checking in progress...
project/node_modules/nest-typed-config/lib/loader/file-loader.ts:90
throw new Error(
^
Error: Environment variable is not set for variable name: 'NODE_ENV'
at replace (project/node_modules/nest-typed-config/lib/loader/file-loader.ts:90:13)
at String.replace (<anonymous>)
at placeholderResolver (project/node_modules/nest-typed-config/lib/loader/file-loader.ts:100:19)
at project/node_modules/nest-typed-config/lib/loader/file-loader.ts:147:30
at Function.getRawConfig (project/node_modules/nest-typed-config/lib/typed-config.module.ts:76:12)
at Function.forRoot (project/node_modules/nest-typed-config/lib/typed-config.module.ts:21:28)
at Object../src/app.module.ts (project/dist/main.js:79:51)
at __webpack_require__ (project/dist/main.js:937:42)
at Object../src/main.ts (project/dist/main.js:700:22)
at __webpack_require__ (project/dist/main.js:937:42)
No errors found.
returning an undefined
without throwing would be fantastic
# .env.yml
testEnvs: ${NODE_ENV}
// in config schema
import { IsOptional, IsString } from 'class-validator';
export class Config {
@IsString()
@IsOptional()
public readonly testEnvs?: string;
}
// in app.module.ts module imports
TypedConfigModule.forRoot({
schema: Config,
load: fileLoader({
ignoreEnvironmentVariableSubstitution: false,
}),
isGlobal: true,
}),
It would be better to respect the config schema, and allow optional fields to be treated as optional.
Nest version: 9.1.1
For Tooling issues:
- Node version: v18.8.0
- Platform: Linux Ubuntu
Others:
[ ] Regression
[ ] Bug report
[x] Feature request
[ ] Documentation issue or request
[ ] Support request
I'm trying to replicate the same configuration capabilities as large open-source projects such as GitLab, Sourcegraph, or Kibana. In those projects, you have the ability to specify configuration via a config file (gitlab.yml, sourcegraph.json, kibana.yml) or an equivalent environment variable (with env variables having priority over the config file).
GitLab | Sourcegraph | Kibana |
---|---|---|
gitlab.yml | admin/config/site.schema.json | kibana.yml |
Env variables | Env variables | Env variables |
So far so good, by following the README, I was able to get to the same setup. The last thing I'm unable to replicate is the ability to have an env variable name that is different from a YML property, but still have the two match and reference the same configuration property. The main reason is that env variables are typically written in uppercase with underscores (SERVER_NAME
) while YML or JSON properties are usually in lowercase & camelCase (serverName
or server.name
).
There could be a few ways to implement something like this:
serverName
to SERVER_NAME
or database.url
to DATABASE_URL
// config.ts
export class Config {
@EnvVariable('SERVER_NAME')
public readonly serverName!: string;
}
The second solution is inspired by something I was able to do with a Golang config library: https://github.com/gravitational/configure
type Config struct {
StringVar string `env:"STRING_VAR" cli:"string-var" yaml:"string_var"`
BoolVar bool `env:"BOOL_VAR" cli:"bool_var" yaml:"bool_var"`
IntVar int `env:"INT_VAR" cli:"int_var" yaml:"int_var"`
HexVar hexType `env:"HEX_VAR" cli:"hex_var" yaml:"hex_var"`
MapVar map[string]string `env:"MAP_VAR" cli:"map_var" yaml:"map_var,flow"`
SliceMapVar []map[string]string `env:"SLICE_MAP_VAR" cli:"slice_var" yaml:"slice_var,flow"`
}
An other Golang config library (https://github.com/peterbourgon/ff) opted for the first solution:
Additionally, the example will look in the environment for variables with a MY_PROGRAM prefix. Flag names
are capitalized, and separator characters are converted to underscores. In this case, for example,
MY_PROGRAM_LISTEN_ADDR would match to listen-addr.
[ ] Regression
[x] Bug report
[ ] Feature request
[ ] Documentation issue or request
[ ] Support request
Using this nest-typed-config in nestjs monorepo, which are built with web pack results. In class-validator
library not being found due to dynamic loading. Previous version exhibited similar behaviour for dotenv
, but it was easier to work around by providing own loader.
Webpack builds should not fail.
Nest version: 9.0.0
For Tooling issues:
- Node version: 16
Others:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.