Comments (7)
It is fine to repeat code when it is 2 LOC. If that still bothers, you can do:
def fossil_fuels, do: read!("fossil_fuels.csv")
And the duplication is effectively gone while being clear what is the docs, what is the function name, etc.
For example, what happens if we start removing the duplication and then we want to add specs or doc metadata? Or we want to add some post-processing to one of the datasets?
from explorer.
For example, what happens if we start removing the duplication and then we want to add specs or doc metadata? Or we want to add some post-processing to one of the datasets?
So, my assumption is that all datasets are similar and needs to be managed in the same manner. So, if we want to add specs, they would all have the same spec which would be hardcoded in the generation. In the case of docs, it would be read from thedocs.md
file.
Won't a dataset be processed before being added to this repo?
However, I get your point of a special case for one dataset. In this case, I would remove this dataset from the list to iterate and write the dataset function by hand.
And the duplication is effectively gone
Another solution is writing a private function with the dataset name as parameter to load the file, and then writing each function by hand.
from explorer.
In the case of docs, it would be read from thedocs.md file.
Docs metadata are Elixir code, so we would need to start parsing the .md
file and evaluating Elixir code. The point is exactly that if your duplication is 2 LOC, that is mostly declaring stuff with docs, specs, and not really any logic, then attempting to remove the duplication is most likely just adding indirection. For example, everyone expects docs to be in the @doc
annotation, not really in .md
, and so on.
EDIT: to further add to this, I don't think the plan is to have many datasets as part of this repo either, especially as scidata is meant to cover the larger ones.
from explorer.
Docs metadata are Elixir code, so we would need to start parsing the .md file and evaluating Elixir code. The point is exactly that if your duplication is 2 LOC, that is mostly declaring stuff with docs, specs, and not really any logic, then attempting to remove the duplication is most likely just adding indirection. For example, everyone expects docs to be in the
@doc
annotation, not really in .md, and so on.
I meant reading the docs.md file at compile time and assigning it to the @doc
annotation. So:
@doc unquote(read!("datasets/#{name}/docs.md"))
EDIT: to further add to this, I don't think the plan is to have many datasets as part of this repo either, especially as scidata is meant to cover the larger ones.
What do you think of this:
def fossil_fuels, do: read_dataset("fossil_fuels")
defp read_dataset(name) do
@datasets_dir
|> Path.join(name <> ".csv")
|> Dataframe.read_csv!()
end
from explorer.
I meant reading the docs.md file at compile time and assigning it to the @doc annotation. So:
I understood. What I mean in the context of someone reading the source code later on, they wouldn't expect the docs in a markdown file.
What do you think of this:
Right, that's pretty much what I had in mind for read!
. :)
from explorer.
I understood. What I mean in the context of someone reading the source code later on, they wouldn't expect the docs in a markdown file.
Yes that's an issue.
Right, that's pretty much what I had in mind for read!. :)
Shall I submit a pr then?
from explorer.
It is fine by me. I don't mind the current code but extracting it also works.
from explorer.
Related Issues (20)
- Precompiled NIFs for freebsd HOT 2
- Add parse datetime from string HOT 1
- add DataFrame.frequencies/2 HOT 1
- Q: How to specify that a number is an epoch when reading a Parquet file? HOT 4
- Normalise errors to return `Exception.t()` from the backend HOT 6
- Document `:backend` and `:lazy` options for `from_*` IO functions HOT 1
- Normalize IO dataframe operations to return {:error, Exception.t}
- `mix ci` difficulties on MacOS as an Elixir beginner HOT 2
- `Explorer.Series.cut` crashes when series is a dataframe field HOT 1
- Add binding to str.slice and/or str.split and/or add trim/2 HOT 6
- `DataFrame.from_csv` incorrectly reads "NA" as `nil` HOT 2
- [Question] Linting Elixir code as an Elixir beginner HOT 1
- Pairwise operations HOT 2
- mutate only certain rows based on a filter? HOT 5
- It is not clear how to filter on categories HOT 3
- Aggregate a series to list in groupby HOT 2
- Missing behaviour on filtering DataFrames on categories. HOT 3
- &Explorer.DataFrame.print/2 does not print header names if no options is provided HOT 4
- Pass dtype to Series callbacks
- Use sink cloud versions of ipc HOT 7
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from explorer.