Library entirely ignore required field in `components/schemas` while merging required fields about modelina

[czlowiek488]

I just figured out that this issue shows only when you use allOf and you define required in both reference and the second object.

[jonaslagoni]

Thanks for the issue @czlowiek488, let me check 👀

[jonaslagoni]

With this AsyncAPI document:

{
  "asyncapi": "2.0.0",
  "info": {
    "title": "example",
    "version": "0.1.0"
  },
  "channels": {
    "example": {
      "publish": {
        "message": {
          "payload": {
            "$id": "someEventName",
            "allOf": [
              {
                "$ref": "#/components/schemas/EventEnvelope"
              },
              {
                "type": "object",
                "properties": {
                  "data": {
                    "type": "object",
                    "properties": {
                      "project": {
                        "$ref": "#/components/schemas/Project"
                      }
                    }
                  }
                }
              }
            ]
          }
        }
      }
    }
  },
  "components": {
    "schemas": {
      "EventEnvelope": {
        "$id": "EventEnvelope",
        "description": "Envelope shared between messages",
        "type": "object",
        "required": [
          "eventName"
        ],
        "properties": {
          "eventName": {
            "type": "string"
          }
        }
      },
      "Project": {
        "$id": "Project",
        "type": "object",
        "required": [
          "id"
        ],
        "properties": {
          "id": {
            "type": "string"
          }
        }
      }
    }
  }
}

Which returns 3 interfaces
SomeEventName - Here <anonymous-schema-3> is a bug, gonna add another issue for this, the current workaround is to specify $id for last schema in allOf.

export interface SomeEventName {
  eventName: string;
  data?: <anonymous-schema-3>;
}

Project

export interface Project {
  id: string;
}

<anonymous-schema-3>

export interface AnonymousSchema_3 {
  project?: Project;
}

Is this not the expected behavior @czlowiek488? besides <anonymous-schema-3> of course.

[czlowiek488]

Oh sorry. I found an issue but I written wrong case to reproduce.
Try this.

components:
  schemas:
    Project:
      $id: Project
      type: object
      properties:
        id:
          type: string
        name:
          type: string


  messages:
        someEventName:
          required: [id]
          $ref: "#/components/schemas/Project"
        
        someEventName2:
          required: [id, name]
          $ref: "#/components/schemas/Project"

[jonaslagoni]

@czlowiek488 that is not a valid AsyncAPI document, messages cannot define required, are you able to provide the full example? I don't care how long it is, potentially you could use https://gist.github.com/

Cause the examples I recreate, all works as expected

[jonaslagoni]

Alright I think I recreated it 🤔

{
  "asyncapi": "2.0.0",
  "info": {
    "title": "example",
    "version": "0.1.0"
  },
  "channels": {},
  "components": {
    "messages": {
      "someEventName": {
        "payload": {
          "required": ["id"],
          "$ref": "#/components/schemas/Project"
        }
      },
      "someEventName2": {
        "payload": {
          "required": ["id", "name"],
          "$ref": "#/components/schemas/Project"
        }
      }
    },
    "schemas": {
      "Project": {
        "$id": "Project",
        "type": "object",
        "properties": {
          "id": {
            "type": "string"
          },
          "name": {
            "type": "string"
          }
        }
      }
    }
  }
}

Only output

export interface Project {
  id: string;
  name: string;
}

Where both fields are required.

[czlowiek488]

This is what I am dealing with https://github.com/xtrf/XTRFCloudInternalEventBus/blob/main/spec/TMSBusinessEvents.yaml

[jonaslagoni]

Have you added $id to the project schema? In order to generate a model named Project? 🤔

[jonaslagoni]

Here is how I see the problem (and I am assuming that you have added the $id property to the project schema).

When a schema is referenced multiple places and the referenced schema has $id sat, such as:

...
      "someEventName": {
        "payload": {
          "required": ["id"],
          "description": "test"
          "$ref": "#/components/schemas/Project"
        }
      },
      "someEventName2": {
        "payload": {
          "required": ["id", "name"],
          "$ref": "#/components/schemas/Project"
        }
      }
...

The Project schema is rendered by accumulating properties i.e. it will inherit description and required from the someEventName payload. It will then merge the schema payload from someEventName2 which means the extra required property name is added to the list.

Temporary workaround: Add the $id from the referenced schemas and do not add an $id to Project.

...
      "someEventName": {
        "payload": {
          "$id": "someEventNameProject",
          "required": ["id"],
          "description": "test"
          "$ref": "#/components/schemas/Project"
        }
      },
      "someEventName2": {
        "payload": {
          "$id": "someEventName2Project",
          "required": ["id", "name"],
          "$ref": "#/components/schemas/Project"
        }
      }
...
      "Project": {
        "type": "object",
        "properties": {
          "id": {
            "type": "string"
          },
          "name": {
            "type": "string"
          }
        }
      }
...

This will force the generator library to generate 2 models for project.

[jonaslagoni]

I will have to ponder some more about what we can do about the underlying solution to make this as clean as possible... Since in principle, you do want two models generated 🤔 Right @czlowiek488 ?

[czlowiek488]

Yes, but this way may be actually the right way. If it could work in someway without $id each $ref, it has to had automatically generated names. And that`s something what should be avoided imho.
I think some kind of error should be throwed in case when you overwrite fields from schema without a $id. Or at least user should be warned.

[jonaslagoni]

If it could work in someway without $id each $ref, it has to had automatically generated names. And that`s something what should be avoided imho.

Yea... I was thinking about whether we could rename the schema using the $id in some context. If there is a clash when referenced from properties we could give it the model the name <property name><schema $id>... Have to think some more about this.

I think some kind of error should be throwed in case when you overwrite fields from schema without a $id. Or at least user should be warned.

Agreed, #152 or a follow-up issue should add this.

[czlowiek488]

Yea... I was thinking about whether we could rename the schema using the $id in some context. If there is a clash when referenced from properties we could give it the model the name <property name><schema $id>... Have to think some more about this.

Ignore it for now, it will complicate code a lot. It should not be included in MVP.

[czlowiek488]

Not to mention that this whole codebase is soooo complex i would rather rewrite it from scratch than maintain....

[jonaslagoni]

Not to mention that this whole codebase is soooo complex i would rather rewrite it from scratch than maintain....

Are you referring to this library? 🤔 If so, then yea... It is unfortunately just the nature of converting JSON Schema validation rules to a model definition (the process we call simplification), even with a rewrite the complexity behind this is unavoidable (in my opion at least).. Especially since those validation rules can be defined in multiple ways...

[czlowiek488]

Yes I do refer to generator-model-sdk, those classes, callbacks, circular function executions makes it hard to maintain.
Main problem is that you have to know how everything works to understand how small piece like function works...
Also simplifier is so hard to understand that you can`t even write right documentation for it.

[jonaslagoni]

Yeaa... @czlowiek488 would you mind creating an issue for this? @magicmatatjahu and I have discussed it before, none of us could see how to change the implementation to make it more maintainable. However, it would be great to have an issue stating it from someone who hasn't build the library 😄

[czlowiek488]

I don`t have time until middle of may. For now I can show you few themes you can read about and what imho will help you to create better code structures.

• Composition pattern - long story short you describe "how" instead of "what", this is huge change in thinking how to develop stuff
• Currying - allow you to instance object without using classes ( no this === no problem )
• async/await - allow you to almost completely remove callbacks, but if you must you should create your own Promise instance where you encapsulate callback.

PS. Classes are the biggest disaster in javascript, only reasons to use it are ( very rarely, should be avoided ) extending and decorators.

[jonaslagoni]

@czlowiek488 no worries, just wanted you to create a general issue about maintainability, you dont have to make suggestions on how to improve it, just raise the issue.

Since this issue have been "solved" or through other issues I am closing it. Feel free to reopen if that is not the case.