srohatgi / graph Goto Github PK
View Code? Open in Web Editor NEWResource management using graphs
License: MIT License
Resource management using graphs
License: MIT License
Currently, createSync()
returns as soon as a resource execution errors out.
This leaves out a possibility that other alternate resources and associated resource sub-trees could still execute. For deeply nested and broad sub-trees this would be ensuring that all resources that can be executed are always executed.
Currently, buildCache
is not persisted between execution runs. This results in the entire graph being executed, with potentially long running checks. While this ensures that the graph execution is always consistent, it can be slow to execute deeply nested graphs.
If we persist the buildCache
, and have an associated policy to refresh it either on demand OR on a time basis, it can avoid potentially expensive checks.
The current implementation of Sync() takes in Resource
slice, with resource names being unique. It may be better to enforce this requirement by changing the signature to take in a map of resources indexed by a resource name.
The new signature would be:
func Sync(resources map[string]Resource, toDelete bool, factory Factory) error
If I want to say A should happen before B, but B does not depend on the output of A, there is no way to specify this.
One needs to specify fromfield and tofield as well.
dependencies:
- fromresource: maestro-search-cfn
fromfield: SHIAMRoleARN
tofield: IAMRoleARN
Remove this requirement.
Currently, the graph lib synchronizes resource graphs based on a correct dependency order. Following validation is performed:
Missing dependencies on a resource graph is not validated. This enhancement proposes adding validation capability for a given graph kind
.
Proposed api changes:
// Opts captures customizable functionality like logging
type Opts struct {
CustomLogger func(args ...interface{})
Decorator func(r Resource) Resource
// Validator captures constrained dependency graph kinds
Validator map[string][]Resource
}
// Sync method uses the Resource slice to generate a DAG. The DAG is verified by comparing it to the
// `kind` parameter. The DAG is processed based on the value of toDelete flag. Resources may be
// processed concurrently. Processed resources may return a status string and or an error. The function
// collects these and aggregates them in respective maps keyed by resource names.
func (lib *Lib) Sync(ctxt context.Context, kind string, resources []Resource, toDelete bool) (map[string]string, error)
this can be reproed by changing spec and override spec in example_usage.go to the following:
const spec = `
kinesis:
- name: mykin2
shardcount: 5
streamname: myEventStream
deployment:
- name: mydep
dependencies:
- fromresource: mykin
fromfield: Arn
tofield: KinesisArn
dynamo:
- name: mydyn
tablename: myDynamoTable
`
const overrideSpec = `
kinesis2:
- name: mykin2
shardcount: 10
`
stack trace:
panic: reflect: call of reflect.Value.Elem on zero Value [recovered]
panic: reflect: call of reflect.Value.Elem on zero Value
goroutine 1 [running]:
testing.runExample.func2(0xbf1f59ffb90317c0, 0x584079, 0x12a8cc0, 0xc00000c070, 0xc00000c018, 0xc0000707e0, 0x1189dd3, 0xd, 0x1193930, 0x119344d, ...)
/usr/local/Cellar/go/1.11.5/libexec/src/testing/example.go:116 +0x6f8
panic(0x115c700, 0xc00000aa40)
/usr/local/Cellar/go/1.11.5/libexec/src/runtime/panic.go:513 +0x1b9
reflect.Value.Elem(0x0, 0x0, 0x0, 0xc000016900, 0xa, 0x1151300)
/usr/local/Cellar/go/1.11.5/libexec/src/reflect/value.go:805 +0x1bd
github.com/srohatgi/graph.checkField(0x0, 0x0, 0xc0000168e8, 0x3, 0x12c5900, 0x0)
/Users/mehulg/go/src/github.com/srohatgi/graph/builder.go:150 +0xea
github.com/srohatgi/graph.check(0xc00000aa20, 0x2, 0x2, 0xc00000ee40, 0xc000095ac0)
/Users/mehulg/go/src/github.com/srohatgi/graph/builder.go:136 +0x4ce
github.com/srohatgi/graph.Sync(0x11b1960, 0xc00000ee40, 0xc00000aa20, 0x2, 0x2, 0xc00000ee00, 0x11b1960, 0xc00000ee40, 0xc000095b48)
/Users/mehulg/go/src/github.com/srohatgi/graph/builder.go:95 +0x46
github.com/srohatgi/graph_test.Example_usage()
/Users/mehulg/go/src/github.com/srohatgi/graph/example_test.go:114 +0x289
testing.runExample(0x1189dd3, 0xd, 0x1193930, 0x119344d, 0x94, 0x0, 0x0)
/usr/local/Cellar/go/1.11.5/libexec/src/testing/example.go:121 +0x1fd
testing.runExamples(0xc000095ef0, 0x12a1aa0, 0x1, 0x1, 0x1000101)
/usr/local/Cellar/go/1.11.5/libexec/src/testing/example.go:45 +0x1ad
testing.(*M).Run(0xc0000b4080, 0x0)
/usr/local/Cellar/go/1.11.5/libexec/src/testing/testing.go:1035 +0x1fb
main.main()
_testmain.go:66 +0x13d
FAIL github.com/srohatgi/graph 0.022s
stack trace is from v0.5.1
Currently, Sync method calls the next node's update method without it to be ready. To avoid that we have to add the polling inside the update method so the next method dependencies are fulfilled.
Builders should implement IsReady() bool
method and Sync method should call it before calling the next node's update method.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.