GithubHelp home page GithubHelp logo

layout21's Introduction

Dan Fritchman

LinkedIn | GitHub | Twitter | Medium


I had recent occasion to write down most of what I had been doing for the past few years, in the form of a PhD dissertation, An Integrated Circuit Design Framework for Human, Computer, and ML Designers. That includes lots of thoughts about our field, its future, and related hot takes. It has been lauded as "actually kinda good" and "not boring" and "how I (the reader) used to write before there was ChatGPT", even by non-nerds. If you are a non-nerd, I recommend the prologue and first chapter or two. (I suspect the quoted reviewers stopped there.) Only those with a shared special breed of nerdery of will care to read to the end.

Projects

Talks

Other Writing

GenAlpha

LinkedIn says we're in stealth mode, because you guys seem to find that cool? But if you've read this long, you can be in on the secret, we're Generation Alpha (Transistor). If you wanna learn more just reach out.

FAQ

Of late I have been asked with increasing frequency whether I am in fact a cactus.
That would make this site (and especially all the work it describes) way more impressive. And the cactus is named Dan too (after someone I know). But no I look more like this.
Dan the cactus (RIP) fills in as an avatar on sites like this. If you see Dan on some other site, that's probably me.

Is this a personal web site? Or just like a GitHub README page?

It's both.

As of this writing https://dan.fritch.mn redirects to here.
For many years it had been a custom site, with custom styling and various build flows and languages and tools, and I got tired of it. GitHub's markdown styling and rendering works just great for stuff like this thanks.


layout21's People

Contributors

colepoirier avatar dan-fritchman avatar davidaguilo avatar growly avatar tannerrogalsky avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

layout21's Issues

Contributing

Hi Dan,

I was investigating porting the established (but limited in capabilities to designs of <1M gates, whereas I’m interested in multi billion and even trillion gate designs as that’s what is needed contemporarily) opensource layout tools in coriolis and alliance from the Sorbonne (http://coriolis.lip6.fr/) to rust, but since you’re already creating a clean room PNR systems and related crates I thought my efforts might me better spent collaborating with you/contributing to your project.

Is this something you would be open to? If so what are some relevant details of the project structure and the way you want development on the project to be done? I guess something to the effect of what some projects will outline in a file named ‘architecture.md’.

Deriving Enum for EnumMap on GdsElement

Problem Statement

Hi,
I'm new to Rust, and am using this library to work on a project. For the project I want to be able to use GdsElement as a key in some sort of dictionary like mapping, I think EnumMap is the way to go.

Reasoning

https://stackoverflow.com/questions/63093321/why-does-rust-prevent-implementing-an-external-trait-for-an-external-struct

Suggests using the newtype idiom, however, applying that to a struct wrapper of enum GdsElement led to the same problem where the enum GdsElement required the Enum trait.

Solution

Adding enum_map as a dependency for gds21 and adding the Enum trait to the derive statement of GdsElement

use enum_map::Enum;

...

#[derive(derive_more::From, Debug, Clone, Deserialize, Serialize, PartialEq, Enum)]
pub enum GdsElement {
    GdsBoundary(GdsBoundary),
    GdsPath(GdsPath),
    GdsStructRef(GdsStructRef),
    GdsArrayRef(GdsArrayRef),
    GdsTextElem(GdsTextElem),
    GdsNode(GdsNode),
    GdsBox(GdsBox),
}

This could also simplify the code for the GdsStruct stats function

Please let me know if this is a silly idea, I'm still new to the language and am learning. I have found your library quite helpful!

Fail to open if an element includes too many points

Thanks for creating this crate.
I've run into a problem that if I created an element with a vast number of points, Klayout would alert that invalid record length (less than 4).
For example

    use gds21::*;
    let mut lib = GdsLibrary::new("mylib");
    let mut newcell = GdsStruct::new("mycell");
    let n=100000;
    newcell.elems.push(GdsElement::GdsBoundary(GdsBoundary {
        layer: 0,
        datatype: 0,
        xy: (1..=n).into_iter().chain(vec![100,150,1,2].into_iter()).collect(),
        ..GdsBoundary::default()
    }));
    lib.structs.push(newcell);
    lib.save("example.gds")?;
    Ok(())

this would work with n=10000, but fail with n=100000.

I was trying to draw a circle, and the circle generated will fail even with n=10000 (n=5000 works).

    use gds21::*;
    use ndarray::{array, Array};
    use std::f64::consts::PI;
    let mut lib = GdsLibrary::new("mylib");
    let mut newcell = GdsStruct::new("mycell");
    let user_unit = 1e-6;
    let db_unit = 1e-9;
    let scale = user_unit / db_unit;
    let units = GdsUnits::new(1. / scale, db_unit);
    lib.units = units;
    let circle_radius = 300.;
    //let resolution = 100e-3;
    //let n=(2. * PI * circle_radius / resolution) as usize;
    let n = 10000;
    let arg_list = Array::linspace(0., 2. * PI, n + 1);
    let point_list =
        arg_list.map(|arg| array![(arg.cos() * circle_radius), (arg.sin() * circle_radius),]);
    newcell.elems.push(GdsElement::GdsBoundary(GdsBoundary {
        layer: 0,
        datatype: 0,
        xy: point_list
            .iter()
            .flatten()
            .copied()
            .map(|x| (x * scale) as i32)
            .collect(),
        ..GdsBoundary::default()
    }));
    lib.structs.push(newcell);
    lib.save("example.gds")?;
    Ok(())

I want to know that if this is a limit of GDSII or it's a bug that you could fix?

`GdsDateTime` validity and formatting

The kind reporters of #19 shared a GDSII file, apparently from an open-source PDK, which several popular layout programs (Klayout, The LayoutEditor) treat as valid, and Gds21 does not. Gds21 panics with an invalid or out-of-range date error while reading it.

Brings up a few things re: handling these dates & times:

  • Above all, we shouldn't panic.
    • If we are going to continue using chrono::NaiveDate, move to the from_ymd_opt constructor which returns a Result instead of panic'ing.
  • GDSII's date & time formatting is not completely clear. The documentation of BGNLIB and BGNSTR says:
Two-Byte Signed Integer
Contains last modification time of library (two bytes
each for year, month, day, hour, minute,and second)
as well as time of last access (same format) and
marks beginning of library. Refer to Figure 4-1. 

And later, in an example:

The 24 bytes of information
following the first two header words contain the modification and last access
date and time. The last 6 words of information, for example, contain: the
year - 85 (55 hex), the month - September (9 hex), the day - 3 (3 hex), the
hour - 10 a.m. (A hex), the minute - 16 (10 hex) and the seconds - o. All
together this record says that this library was last modified on September 3,
1985 at 10:16:00 a.m. 

A few things to glean from this:

  • Years are referenced to 1900
  • Days are valued 1-31
  • Months are valued 1-12
  • Hours are valued 1-12

Now, what is to be done with invalid values, e.g. "month 30" or "hour 25"? The spec does not say.

Perhaps more relevant: should we care?
The file reported in #19 sets a few such invalid values, and those other programs accept them. They don't have an obvious place to find these date/ time fields reported in the UI anywhere.

My general thoughts:

  • When we create GDSII anew, we should make a best-effort to write these dates, and respect GDSII's format.
    • Thus far we are failing to meet that format in one respect: the "1900 referencing".
  • When reading existing files, we probably shouldn't fail for sake of these dates.
  • Propose making GdsDateTime a two-variant enum, with:
    • A (valid) DateTime when possible, or when created anew, and
    • A "raw" Bytes comprising those six words when not.

RUSTSEC-2020-0159: Potential segfault in `localtime_r` invocations

Potential segfault in localtime_r invocations

Details
Package chrono
Version 0.4.19
URL chronotope/chrono#499
Date 2020-11-10

Impact

Unix-like operating systems may segfault due to dereferencing a dangling pointer in specific circumstances. This requires an environment variable to be set in a different thread than the affected functions. This may occur without the user's knowledge, notably in a third-party library.

Workarounds

No workarounds are known.

References

See advisory page for additional details.

Support/ Scale Arbitrary GDS Units

Thus far we have the tacit assumption that importable GDSII's have one of a handful of values for their GdsUnits: 1nm, 1µm, etc.
In practice this has generally been the case. Until it ain't. And on some recently encountered examples, it ain't.
Layout21's internal units can retain the relatively coarse order-of magnitude setting, so long as we scale GDS coordinates, probably around here:

fn import_units(&mut self, units: &gds21::GdsUnits) -> LayoutResult<Units> {

GDS loading could panick at "invalid or out-of-range date".

Really glad to find a layout data parsing project based on rust. It's really good and brings a lot of convenience and fun. I want to mention a problem I met when I use this library.

I try to read some gds library files. My code:

use gds21::{GdsLibrary, GdsStruct};
#[test]
fn gds_loading() {
    let lib = GdsLibrary::load("assets/data/gds/AND2X2.gds");
    dbg!(lib.unwrap());
}

And it panicked, showing: thread 'gds_loading' panicked at 'invalid or out-of-range date'.

I think this could be related to GDS datetime. And when I test on other libaray gds files it could happen too. I haven't digged it deep yet. If more info is needed please let me know.

Environment:

  • rust: 1.63
  • gds21: 0.2.0
  • on pop-os 22.04

Bests.

RUSTSEC-2021-0073: Conversion from `prost_types::Timestamp` to `SystemTime` can cause an overflow and panic

Conversion from prost_types::Timestamp to SystemTime can cause an overflow and panic

Details
Package prost-types
Version 0.6.1
URL tokio-rs/prost#438
Date 2021-07-08
Patched versions >=0.8.0

Affected versions of this crate contained a bug in which untrusted input could cause an overflow and panic when converting a Timestamp to SystemTime.

It is recommended to upgrade to prost-types v0.8 and switch the usage of From&lt;Timestamp&gt; for SystemTime to TryFrom&lt;Timestamp&gt; for SystemTime.

See #438 for more information.

See advisory page for additional details.

All these `clap::Parser` warnings in the `layout21converters` CLIs

It seems they really really want us to update usage of many of these Parser fields:

warning: use of deprecated unit variant `clap::ArgAction::StoreValue`: Replaced with `ArgAction::Set` or `ArgAction::Append`
  --> layout21converters/src/gds2ser.rs:18:5
   |
18 |     gds: String,
   |     ^^^
   |
   = note: `#[warn(deprecated)]` on by default

warning: use of deprecated unit variant `clap::ArgAction::StoreValue`: Replaced with `ArgAction::Set` or `ArgAction::Append`
  --> layout21converters/src/gds2ser.rs:21:5
   |
21 |     fmt: String,
   |     ^^^

warning: use of deprecated unit variant `clap::ArgAction::StoreValue`: Replaced with `ArgAction::Set` or `ArgAction::Append`
  --> layout21converters/src/gds2ser.rs:24:5
   |
24 |     out: String,
   |     ^^^

warning: use of deprecated associated function `clap::ArgMatches::is_present`: Replaced with either `ArgAction::SetTrue` or `ArgMatches::contains_id(...)`
  --> layout21converters/src/gds2ser.rs:27:14
   |
27 |     verbose: bool,
   |              ^^^^

warning: use of deprecated associated function `clap::Arg::<'help>::validator`: Replaced with `Arg::value_parser(...)`
  --> layout21converters/src/gds2ser.rs:18:5
   |
18 |     gds: String,
   |     ^^^

warning: use of deprecated associated function `clap::Arg::<'help>::validator`: Replaced with `Arg::value_parser(...)`
  --> layout21converters/src/gds2ser.rs:21:5
   |
21 |     fmt: String,
   |     ^^^

warning: use of deprecated associated function `clap::Arg::<'help>::validator`: Replaced with `Arg::value_parser(...)`
  --> layout21converters/src/gds2ser.rs:24:5
   |
24 |     out: String,
   |     ^^^

warning: function `main` is never used
  --> layout21converters/src/gds2ser.rs:30:4
   |
30 | fn main() -> Result<(), Box<dyn Error>> {
   |    ^^^^
   |
   = note: `#[warn(dead_code)]` on by default

warning: function `parse_format` is never used
  --> layout21converters/src/gds2ser.rs:37:4
   |
37 | fn parse_format(format: &str) -> Result<SerializationFormat, Box<dyn Error>> {
   |    ^^^^^^^^^^^^

   Compiling layout21converters v0.2.1 (/Users/dan/dev/Vlsir/Layout21/layout21converters)
warning: `layout21converters` (lib) generated 9 warnings
warning: use of deprecated unit variant `clap::ArgAction::StoreValue`: Replaced with `ArgAction::Set` or `ArgAction::Append`
  --> layout21converters/src/lef2yaml.rs:16:5
   |
16 |     lef: String,
   |     ^^^
   |
   = note: `#[warn(deprecated)]` on by default

warning: use of deprecated unit variant `clap::ArgAction::StoreValue`: Replaced with `ArgAction::Set` or `ArgAction::Append`
  --> layout21converters/src/lef2yaml.rs:19:5
   |
19 |     yaml: String,
   |     ^^^^

warning: use of deprecated unit variant `clap::ArgAction::StoreValue`: Replaced with `ArgAction::Set` or `ArgAction::Append`
  --> layout21converters/src/proto2gds.rs:18:5
   |
18 |     proto: String,
   |     ^^^^^
   |
   = note: `#[warn(deprecated)]` on by default

warning: use of deprecated unit variant `clap::ArgAction::StoreValue`: Replaced with `ArgAction::Set` or `ArgAction::Append`
  --> layout21converters/src/gds2proto.rs:17:5
   |
17 |     gds: String,
   |     ^^^
   |
   = note: `#[warn(deprecated)]` on by default

warning: use of deprecated unit variant `clap::ArgAction::StoreValue`: Replaced with `ArgAction::Set` or `ArgAction::Append`
  --> layout21converters/src/proto2gds.rs:20:5
   |
20 |     gds: String,
   |     ^^^

warning: use of deprecated unit variant `clap::ArgAction::StoreValue`: Replaced with `ArgAction::Set` or `ArgAction::Append`
  --> layout21converters/src/proto2gds.rs:22:5
   |
22 |     tech: String,
   |     ^^^^

warning: use of deprecated unit variant `clap::ArgAction::StoreValue`: Replaced with `ArgAction::Set` or `ArgAction::Append`
  --> layout21converters/src/gds2proto.rs:20:5
   |
20 |     proto: String,
   |     ^^^^^

warning: use of deprecated associated function `clap::ArgMatches::is_present`: Replaced with either `ArgAction::SetTrue` or `ArgMatches::contains_id(...)`
  --> layout21converters/src/lef2yaml.rs:22:14
   |
22 |     verbose: bool,
   |              ^^^^

warning: use of deprecated associated function `clap::Arg::<'help>::validator`: Replaced with `Arg::value_parser(...)`
  --> layout21converters/src/lef2yaml.rs:16:5
   |
16 |     lef: String,
   |     ^^^

warning: use of deprecated associated function `clap::Arg::<'help>::validator`: Replaced with `Arg::value_parser(...)`
  --> layout21converters/src/lef2yaml.rs:19:5
   |
19 |     yaml: String,
   |     ^^^^

warning: use of deprecated associated function `clap::ArgMatches::is_present`: Replaced with either `ArgAction::SetTrue` or `ArgMatches::contains_id(...)`
  --> layout21converters/src/proto2gds.rs:24:14
   |
24 |     verbose: bool,
   |              ^^^^

warning: use of deprecated associated function `clap::ArgMatches::is_present`: Replaced with either `ArgAction::SetTrue` or `ArgMatches::contains_id(...)`
  --> layout21converters/src/gds2proto.rs:23:14
   |
23 |     verbose: bool,
   |              ^^^^

warning: use of deprecated associated function `clap::Arg::<'help>::validator`: Replaced with `Arg::value_parser(...)`
  --> layout21converters/src/proto2gds.rs:18:5
   |
18 |     proto: String,
   |     ^^^^^

warning: use of deprecated associated function `clap::Arg::<'help>::validator`: Replaced with `Arg::value_parser(...)`
  --> layout21converters/src/proto2gds.rs:20:5
   |
20 |     gds: String,
   |     ^^^

warning: use of deprecated associated function `clap::Arg::<'help>::validator`: Replaced with `Arg::value_parser(...)`
  --> layout21converters/src/proto2gds.rs:22:5
   |
22 |     tech: String,
   |     ^^^^

warning: use of deprecated associated function `clap::Arg::<'help>::validator`: Replaced with `Arg::value_parser(...)`
  --> layout21converters/src/gds2proto.rs:17:5
   |
17 |     gds: String,
   |     ^^^

warning: use of deprecated associated function `clap::Arg::<'help>::validator`: Replaced with `Arg::value_parser(...)`
  --> layout21converters/src/gds2proto.rs:20:5
   |
20 |     proto: String,
   |     ^^^^^

warning: `layout21converters` (bin "proto2gds" test) generated 7 warnings
warning: `layout21converters` (bin "gds2ser" test) generated 7 warnings (7 duplicates)
warning: `layout21converters` (lib test) generated 8 warnings (8 duplicates)
warning: `layout21converters` (bin "gds2proto" test) generated 5 warnings
warning: `layout21converters` (bin "lef2yaml" test) generated 5 warnings

CI Tests Down

Likely due to upgrades in Rust versions etc, need to be caught up

TOML `SerializationFormat` fails for `GdsLibrary`

This:

/// Create an empty library with known dates
fn empty_lib() -> GdsLibrary {
    // Create an empty library
    let mut lib = GdsLibrary::new("empty");
    // Set its dates to some known value, so we can check it round-trips
    lib.dates = test_dates();
    // And return it for other test
    lib 
}
#[test]
fn empty_lib_to_toml() -> GdsResult<()> {
    let lib = empty_lib();
    Toml.save(&lib, &resource("empty.gds.toml")).expect("save failed");
    Ok(())
}

Fails with:

failures:

---- tests::empty_lib_to_toml stdout ----
thread 'tests::empty_lib_to_toml' panicked at 'save failed: Error(UnsupportedType)', gds21/src/tests.rs:214:50
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Notes:

  • Yaml & Json both work
  • Yaml, Json, and Toml all work for lef21::LefLibrary

Unclear at this point what the UnsupportedType is.

Unclear when this was injected, as we had generally only been testing one format of serialization (GDS to JSON, LEF to YAML, nothing to TOML). And it wasn't clear that there are value-types supported in YAML & JSON that aren't in TOML(?). Apparently GdsLibrary has one or more of them.

Moar Conversion CLIs

PR #35 added GDS to JSON, YAML, TOML (kinda: see #33), and "choose any of those" CLI programs to layout21converters.

There is a larger & more comprehensive matrix that could be added, including:

  • The inverse of all those - markup-languages to GDS
  • LEF to all those formats
  • All those formats to LEF

The VLSIR types are also Serialize and Deserialize, so can in principle be converted to any of those formats too. Whether we want to is another matter.

Move Canonical Proto-Source to `Vlsir`

Layout21 is part of a family of projects, largely built around and driven by their Protobuf-defined data schemas.
The canonical home for that schema is now https://github.com/dan-fritchman/Vlsir.

Layout21 thus far has a "vendored" (and likely slowly becoming incompatible) version of this schema.
Moving it over will (I think) incur:

  • Adding Rust bindings to the Vlsir repository.
  • Moving to depend on them, generally deprecating layout21protos
  • Cleaning up whatever odds & ends follow

RUSTSEC-2020-0071: Potential segfault in the time crate

Potential segfault in the time crate

Details
Package time
Version 0.1.44
URL time-rs/time#293
Date 2020-11-18
Patched versions >=0.2.23
Unaffected versions =0.2.0,=0.2.1,=0.2.2,=0.2.3,=0.2.4,=0.2.5,=0.2.6

Impact

Unix-like operating systems may segfault due to dereferencing a dangling pointer in specific circumstances. This requires an environment variable to be set in a different thread than the affected functions. This may occur without the user's knowledge, notably in a third-party library.

The affected functions from time 0.2.7 through 0.2.22 are:

  • time::UtcOffset::local_offset_at
  • time::UtcOffset::try_local_offset_at
  • time::UtcOffset::current_local_offset
  • time::UtcOffset::try_current_local_offset
  • time::OffsetDateTime::now_local
  • time::OffsetDateTime::try_now_local

The affected functions in time 0.1 (all versions) are:

  • at
  • at_utc
  • now

Non-Unix targets (including Windows and wasm) are unaffected.

Patches

Pending a proper fix, the internal method that determines the local offset has been modified to always return None on the affected operating systems. This has the effect of returning an Err on the try_* methods and UTC on the non-try_* methods.

Users and library authors with time in their dependency tree should perform cargo update, which will pull in the updated, unaffected code.

Users of time 0.1 do not have a patch and should upgrade to an unaffected version: time 0.2.23 or greater or the 0.3 series.

Workarounds

No workarounds are known.

References

time-rs/time#293

See advisory page for additional details.

Surprise to see Layout21 become much greater

Hi fritchman,
I notice your crate about half-year ago, today I find now Layout21 support the gds2/lef parse/write, very impressive by your work. I am interested in PD & Rust and I write a simple gds2 parser in rust long time ago. Any further plans for this crate? I am willing to help if possible.

python interface

would be cool to create a python binding (i.e. with pyo3). i think you would attract a lot more users that way.

The `Utc::now` RUSTSEC

cargo audit runs such as this one:
https://github.com/dan-fritchman/Layout21/pull/35/checks?check_run_id=10213135939

Have been turning up this advisory about the chrono crate:
https://rustsec.org/advisories/RUSTSEC-2020-0071.html

In which, if you do some fun multi-threaded environment-variable fiddling, it crashes, or sends your credit card numbers to North Korea, or something. After #35 our only usage of chrono, and really any time-related stuff, is calling its now function to get a creation-time for new Librarys. The now function is among those effected by the RUSTSEC.

Rooting around the issue, it appears there's no workaround, and the author is not really supporting the chrono crate. Of all those facts, the last seems the most concerning.

There does not appear to be an obvious, popular replacement for this. The standard library's SystemTime::now returns the integer (seconds, nanoseconds) in the epoch, which would need to be converted into (year, month, day, hour, minute, second) for formats that shall remain nameless, such as GDSII.

Expand LEF Support to include the technology components

The Lef21 docs begin with:

LEF is near-ubiquitously used IC-industry-wide for two related purposes:

LEF design libraries, primarily comprised of LEF macros, provide the physical abstract view of circuit designs.
Such abstract-views are commonly the target for layout-synthesis programs ("place and route").
They include a circuit's pin locations and requirements for physical blockages ("obstructions"), among other metadata, typically without including the circuit's internal implementation.
LEF technology descriptions ("tech-lef") provide a concise description of design-rules for assembling such cells, as commonly performed by layout-synthesis software.

Lef21 includes comprehensive support for parsing and writing LEF design libraries, primarily stored as its [LefLibrary] and [LefMacro] types. A select subset of tech-lef features are also supported, particularly those which blur the lines between technology and library data.

This issue: add the latter "tech-lef" content.
This will be especially valuable for other layout21 use cases which endeavor to auto-generate layout from existing technology data - especially from tech-LEF.


Likely Implementations

  • Lef21's primary "top-level" entity is the LefLibrary, which is essentially the manifestation of the "LEF design library" idea
  • Add two more:
    • LefTech (or similar), which includes all of the technology-specification contents (SITE, LAYER, etc.)
    • A union of LefLibrary and LefTech, which I suppose can just be called Lef.

Some questions about `Rect` and `BoundBox`

Rect and BoundBox

I'm curious if Rect should be constrained to have the same guarantees as BoundBox where p0 is the (x, y) values closest to negative infinity, and p1 is the (x ,y) values closest to positive infinity, then obviating the need for the BoundBox type which is essentially just a Rect which results in code duplication.

In that scenario would it then also make sense to define Rect as follows?

pub struct Rect {
    x_min: Int,
    y_min: Int,
    x_max: Int,
    y_max: Int,
}

Perceived BoundBox footgun

Something that I found to be a bit of a footgun with BoundBox is that 'empty, otherwise invalid [BoundBox]' is defined to have the value Int::MAX for p0, and Int::MIN for p1. Would it be better to define BoundBox as an enum sum type (like std::Option) as follows:

pub enum BoundBox {
    Valid(Rect),
    Invalid
}

Would this add too much overhead to what the BoundBox type is intended for? Does the user need to check the validity of the BoundBox regardless?

Remove `Ptr` and `PtrList`

Ptr and PtrList are useful constructs but they don't exactly match the usage patterns present in this library. I see two problems both relating to the fallible nature of acquiring locks the data:

  1. It makes the API more "noisy". I'm not sure that many of these functions even can fail to acquire a lock since they're almost exclusively reads. But we still hoist the potential for those errors into the API, requiring the user to handle them. Or there are some places where we simply unwrap a lock result. While I think that those unwraps are unlikely to error I hope it illustrates the point.
  2. It makes implementations more verbose as one has consider the guard lifetimes.
    Neither of these are necessarily damning (well, the first more than the second, maybe) but my feeling that they are important is compounded by my perception that we aren't using the interior mutability aspect of a RwLock. All of the places that we acquire a write lock are behind a mutable reference already. It seems to me that we're exclusively using the reference counting aspect of Ptr and PtrList and not the interior mutability at all. As such, perhaps there is a more applicable abstraction.

I also have a sense that if things like the layer maps were modified from outside of the containing data structures there is significant potential for breakage. Or at least it's not clear to me that allowing changes to interior data apart from the API is desired.

Solutions

Replace std::sync::RwLock with parking_lot::RwLock

Pros: Low effort.
Cons: Doesn't really fix point 2 but would improve point 1.

Use a slotmap/petgraph

We're representing a directed graph. Petgraph is well used and loved for good reason. This would solve both problems and potentially allow us to leverage existing patterns for traversal and analysis.
Pros: The arena allows us to maintain cheap, potentially cyclical "references". Mutating only from behind a mutable reference creates patterns that users expect.
Cons: Access to the actual resource requires access to the arena. Functions like Layout::flatten would require a reference to the containing library, etc.

Replace Ptr with arc_swap::ArcSwap

Pros: We're mostly read-heavy so we get to keep our pointer semantics while getting ride of the locks & guards.
Cons: Our individual cells become copy-on-write by necessity.

There are probably other ways of doing this but I think these represent a good start.

My personal inclination would be towards a slotmap/petgraph but I think I would be easily convinced of ArcSwap as well.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.