aivantg / airtable-schema-generator Goto Github PK

Hey, thanks for this repo. I checked it out and it's awesome.

I have a different use case: I just want to dump everything into SQL files and your repo gives me a lot of the functionality I need. I wanted to check if it's OK to fork and modify your code. I don't think you will want many of my edits, as I want to pull out a lot of the code which I don't need (CRUD helpers)

Is this MIT licensed? Either way, I'll link back to this repo in my readme

function createRecords(table, records) {
  const transformedRecords = records.map((record) => ({
    fields: toAirtableFormat(record, table),
  }));
  return base(table)
    .create(transformedRecords)
    .then((newRecords) => {
      return newRecords.map((newRecord) => newRecord.getId());
    })
    .catch((err) => {
      throw err;
    });
}

Corresponding request.js:

export const createManyStores = async (records) => {
  const createPromises = [];
  const numCalls = Math.ceil(records.length / 10);
  for (let i = 0; i < numCalls; i += 1) {
    const subset = records.slice(i * 10, (i + 1) * 10);
    if (subset.length > 0)
      createPromises.push(createRecords(Tables.Stores, subset));
  }
  return Promise.all(createPromises);
};

*Airtable API only allows batches of up to 10

Add exceptions for pluralize/depluralize special cases, because English is weird (e.g News - depluralizing; Feedback - pluralizing)

Currently all linked records (even if they're one-to-one) are stored as arrays and this is annoying to write code with

Right now, any linked record that ends in "S" gets convert to "ids" but this might not be desired. Either make assumptions more explicit in readme or make more intentional transformations

Hello there !

When using "generate-airtable-schema", the user get an error because the selector doesnt seems to be good. (I guess they changed it).

On "airtable-schema-generator/lib/scrapper.js", l25 the selector should be :

try { await page.click( '#sign-in-form-fields-root > div > label > button[type="submit"]' ); } catch (err) { await page.click( '#sign-in-form-fields-root > div > label > input[type="submit"]' ); }

Or something similar.

For the foreign-key-many depluralization, there are rare cases when depluralizing creates bugs:

	clerkFeedbacIds: { name: `Clerk Feedback`, type: `foreignKey-many` }
	productsPurchaseIds: { name: `Products Purchased`, type: `foreignKey-many` },

Fixing this would require a depluralize function similar to the pluralize function

Hello !
Thanks for the work done here.

I'm using get__ById and get__ByIds methods, but it seems the first one is working fine, but not the last one. One workaround was to create a new field at the end of the table called id with a special formula RECORD_ID(), but it seems wrong (Especially with get_byId working)

Any feedback of this ?

My error : error: "INVALID_FILTER_BY_FORMULA", message: "The formula for filtering records is invalid: Unknown field names: id()

Thanks.

Add exceptions for columns and tables that should not be included in the schema (don't yell when schema doesn't include them).

Not sure if this is really needed - the use case for this is when one developer is updating the schema, or testing a new column addition, but testing won't be allowed since the generator wants to enforce that the generator is run and breaks the app while trying to test unrelated changes, e.g

Error converting Products record from Airtable. Could not find column of name "Original Customer Cost" in local copy of schema.js. Please run the schema generator again to get updates

The fix for now is to run the generator, but this will re-generate the schema with a column we don't want to keep since it'll be updated in a separate PR.

A quirk of the Airtable API is that when it returns records, it will only populate fields if they are non-empty, which is pretty annoying.

Ex: Trying to get products by productIds from a store; initializing productIds to an empty array in the fromAirtableFormat might be useful.

This quirk has caused a few small bugs so think it might be worth pre-filling all the fields (since we know what they are from the schema) to be the correct empty type ('', {}, etc).

Currently if you have an array of record IDs, you have to make a unique findRecordById call for each one. Can we make a function that takes in an array of IDs and only makes one fcall?

function updateRecords(table, updatedRecords) {
  const transformedRecords = updatedRecords.map((updatedRecord) => ({
    id: updatedRecord.id,
    fields: toAirtableFormat(updatedRecord.fields, table),
  }));
  return base(table)
    .update(transformedRecords)
    .then((records) => {
      return records[0].id;
    })
    .catch((err) => {
      throw err;
    });
}

Corresponding request.js:

export const updateManyStores = async (recordUpdates) => {
  const updatePromises = [];
  const numCalls = Math.ceil(recordUpdates.length / 10);
  for (let i = 0; i < numCalls; i += 1) {
    const subset = recordUpdates.slice(i * 10, (i + 1) * 10);
    if (subset.length > 0)
      updatePromises.push(updateRecords(Tables.Stores, subset));
  }
  return Promise.all(updatePromises);
};

*Airtable only allows updates in batches of maximum size 10

They're not super well written as of now, unnecessary promises, not fully featured

Example: "News"

Prettier runs in our repo, and always updates the files because the spacing is a little off from snippets.js; also missing semicolons.

I tried to follow the instructions here https://github.com/aivantg/airtable-schema-generator#manual-mode but am getting _ is not defined.

The mostly updated code is:

// Need to handle the foreign key part
copy(Object.entries(application.tablesById).reduce((p,[k, v]) => ({...p, [k]: {...v, sampleRows: undefined, $$hashKey: undefined, columns: v.columns.map(c => ({...c, $$hashKey: undefined}))}}), {}))

I came across this and thought of you! Might help ensure your work doesn't break, since this is maintained by airtable themselves for visualizing the schema in a block:
https://github.com/Airtable/apps-base-schema/blob/17b594aa0c3ed047b2f5d23ad7d1699ca025cf23/frontend/parseSchema.js

(sorry if I misunderstand anything -- have only high-level understanding of what each project does)

Thanks for everything!

Not sure how necessary this is if it can just happen locally, but it would likely be important when access control is implemented

getRecordsByAttribute and getRecordByIds functions implicitly use their own filterByFormula but can both take in sort parameters. getRecordsByIds currently does not have one.