nornir-automation / gornir Goto Github PK

Create a new option WithRunner. See: #13 (comment).

We can do that through SSHOpen.

Follow up from #37 (comment)

The metadata or descriptions are not too relevant for the actual execution.

For pre-defined tasks (like SSHClose) I propose we provide a default description (the user can change it). The user can also create its own tasks:

type SSHOpen struct {
	description string
}

type SSHClose struct {
	description string
}

type MyTask struct {
	description string
}

func NewSSHOpen() SSHOpen {
	return SSHOpen{
		description: "Open SSH connection",
	}

}

func NewSSHClose() SSHClose {
	return SSHClose{
		description: "Close SSH connection",
	}

}

The goal is the user program at very high level looks something like (MyTask in the same connection package for simplicity in this example):

func main() {
	...
	sshOpen := connection.NewSSHOpen()
	sshClose := connection.NewSSHClose()
	myTask := connection.MyTask{description: "Something else"}
	
	gr.RunSync(sshOpen)
	defer gr.RunSync(sshClose)
	gr.RunSync(myTask)

	fmt.Println("Doing this:", sshOpen.description, myTask.description, sshClose.description)
	...

}

The user creates tasks that are then executed in single lines.

Or anything similar to validate the code is still passing all the test cases. I know might the API is still in its early days, but enforcing testing early on will surely pay dividends in the long run.

Hey folks,
is this project alive? 3 months no commits.
If so, could you please provide a Contribution Guidelines? Something similar to the nornir's Contribution Guidelines would be much appreciated.

It'd probably be nice to support the constructor pattern to make things easier to use. For instance, instead of:

gr := &gornir.Gornir{
	Inventory: inventory,
	Logger:    logger,
}

results, err := gr.RunS(
	"What's my ip?",
	runner.Parallel(),
	&task.RemoteCommand{Command: "ip addr | grep \\/24 | awk '{ print $2 }'"},
)

we could do instead:

gr := &gornir.Gornir{
	Inventory: inventory,
	Logger:    logger,
        Runner:    runner.Parallel(),
}

results, err := gr.RunS(
	"What's my ip?",
	&task.RemoteCommand{Command: "ip addr | grep \\/24 | awk '{ print $2 }'"},
)

...

results, err := gr.WithRunner(runner.SomeOtherRunner()).Filter(filterHostsFunc).RunS(
	"What's my ip?",
	&task.RemoteCommand{Command: "ip addr | grep \\/24 | awk '{ print $2 }'"},
)
...

Basically the idea would be to instantiate Gornir with default objects and then use the constructor pattern to override them when needed

Right now we have a logrus logger, it'd be nice to add a null one in case people doesn't want logs.

test examples somehow

The idea is to avoid the pattern:

	results, err := gr.RunSync(
		&task.RemoteCommand{Command: "ip addr | grep \\/24 | awk '{ print $2 }'"},
	)
	if err != nil {
		log.Fatal(err)
	}
	if err := output.RenderResults(os.Stdout, results, true); err != nil {
		log.Fatal(err)
       }

To do that we could either modify existing Logging interface or create a new one like:

type Output interface {
    PreHost(Context, Host, Task)            // Called before executing a task on a host
    PostHost(Context, Host, Task)           // Called before executing a task on a host
    PreTask(Context, Task)                  // Called before executing the task on any host
    PostTask(Context, Task)                 // Called after executing the task on all the hosts
    Success(Context, TaskInstanceResult)    // Called if the task succeeded
    Fail(error)                             // Called if the task failed
}

I think that extending the logging interface should be enough and different plugins could do different things. For instance, a logrus plugin could just ignore those method (no ops) and implement Info, Error, etc., while a RenderResult plugin could do the opposite; implement logging like methods like Info and Error and implement instead Host, Task, Success and Fail.

To make this even more useful, it'd be great to allow multiple output loggers.

An alternative would be to leave things as they are and add a callback to RunAsync but I think there is value on the output interface as it will simplify the overall workflow and delegate responsibility to the output plugin.

I also think it's general enough that users could leverage the pattern to create their own "event based" workflow. For instance, your "Output" plugin could update a database on Success with data the gathered on the task, and send notifications to slack on Fail, PreTask and PostTask.

We need to make it a bit more configurable although the way the interface is defined it might be that everything we need to do is just pass it without the need of a "plugin". Although the plugin might be a convenience function for people with the common use cases.

Hi, I would personally prefer to avoid wrapping the Context in another Context struct for two reasons.

1. Context does provide a mechanism to store values already, however is recommended to "Cnly use context.Value for data that can’t be passed through your program in any other way. In practice, this means only using context.Value for request-scoped information, like request IDs, user authentication tokens, and so on". SOURCE

type Context struct {
	ctx    context.Context
	title  string
	id     string
	gr     *Gornir
	logger Logger
	host   *Host
}

2. You are passing these values explicitly to the runner functions anyways.

func (g *Gornir) RunA(title string, runner Runner, task Task, results chan *JobResult) error {
	err := runner.Run(
		NewContext(context.Background(), title, g, g.Logger),
		task,
		results,
	)
	...
}

Let me know if I'm missing anything. Also, if you embed the context.Context all its methods get promoted so you don't really need to reimplement everything to satisfy the interface.

enabling golint with with golangci-lint requires some extra handling, we need to enable it and fix the issues raised by it. To enable it, make lint should be:

.PHONY: lint
lint:
	docker run \
		--rm \
		-v $(PWD):/go/src/$(PROJECT) \
		-w /go/src/$(PROJECT) \
		golangci/golangci-lint \
			golangci-lint run
	docker run \
		--rm \
		-v $(PWD):/go/src/$(PROJECT) \
		-w /go/src/$(PROJECT) \
		golangci/golangci-lint \
			golangci-lint run --no-config --exclude-use-default=false --disable-all --enable=golint

evaluate if the inventory needs to be able to read the user-defined data and handle var resolution.

The current example is outdated now. Leaving this one for newcomers.

	results, err := gr.RunSync(
		"What's my ip?",
		&task.RemoteCommand{Command: "ip addr | grep \\/24 | awk '{ print $2 }'"},
	)

package main

import (
"context"
"fmt"
"github.com/nornir-automation/gornir/pkg/gornir"
"github.com/nornir-automation/gornir/pkg/plugins/connection"
"github.com/nornir-automation/gornir/pkg/plugins/inventory"
"github.com/nornir-automation/gornir/pkg/plugins/logger"
"github.com/nornir-automation/gornir/pkg/plugins/output"
"github.com/nornir-automation/gornir/pkg/plugins/runner"
"github.com/nornir-automation/gornir/pkg/plugins/task"
"os"
)

func main() {
log := logger.NewLogrus(false)

file := "D:\\Python Project\\Nornir\\inventory\\hosts.yaml"
plugin := inventory.FromYAML{HostsFile: file}
inv, err := plugin.Create()
if err != nil {
	log.Fatal(err)
}

gr := gornir.New().WithInventory(inv).WithLogger(log).WithRunner(runner.Parallel())


results, err := gr.RunSync(
	context.Background(),
	&connection.SSHOpen{},
)
if err != nil {
	log.Fatal(err)
}


defer func() {
	results, err = gr.RunSync(
		context.Background(),
		&connection.SSHClose{},
	)
	if err != nil {
		log.Fatal(err)
	}
}()


results, err = gr.RunSync(
	context.Background(),
	**&task.RemoteCommand{Command: "tftp 10.8.61.116 put vrpcfg.zip **Hostname**.zip"}**,
)
if err != nil {
	log.Fatal(err)
}
// next call is going to print the result on screen
output.RenderResults(os.Stdout, results, "What is my ip?", true)
fmt.Println(gornir.Task.host)

}
In this code, I wish to run command "tftp 10.8.61.116 put vrpcfg.zip Hostname.zip" in every switch , Hostname stands for every switch 's management ip address . I know we can use f'tftp 10.8.61.116 put vrpcfg.zip {task.host.hostname}.zip' in nornir in python but I don't know how to do in gornir . Could you provide any idea ?

Should gornir.New() provide safe defaults?. Should require you to provide mandatory attributes?. Today looks as simple as:

func New() *Gornir {
	return new(Gornir)
}

Even if I add use the context.WithTimeout, but it does not effective

i would love to see a network automation tool that has everything running concurrently,
for example:
you can have two channels JobsChan and ResultsChan, then a readers reads inventory and playbooks and start sending Jobs on 'JobsChan.

on the other side you have the Runner that fetch Jobs and spawn a task that execute them and send the results to ResultsChan.

and finally you have the a the ResultHandler that pull results from ResultsChan and send them to the output fds weather an stdout or file.
do you think this will have added Value? or you think this approach is not really needed in network automation as you don't expect huge inventory/play files?

Today, most of the concrete types use pointer semantics; *Gornir, *Inventory, *Host, etc. This is OK when sharing data that you want to modify, however this means we need to be careful to avoid data races when launching multiple goroutines. We might need to implement mutexes to protect us from this.

We can potentially change most of the types to value semantics to pass copies of the values around. This also means we can store the data in the Stack and not the Heap.

Not sure at this point what the best approach is in this case, but wanted to raise awareness of this so we can address it somewhere along the line.

Right now connections are handled by tasks, we should have gornir do that in a similar fashion to nornir

runners should skip failed hosts unless stated otherwise

Arista switches, by default, allow only publickey and keyboard-interactive auth methods.
Gonir fails to create an ssh connection with the following error:

ssh: handshake failed: ssh: unable to authenticate, attempted methods [none], no supported methods remain

I fixed this error by adding ssh.KeyboardInteractive method to the ssh.ClientConfig:

// Run implements gornir.Task interface
func (t *SSHOpen) Run(ctx context.Context, logger gornir.Logger, host *gornir.Host) (gornir.TaskInstanceResult, error) {
sshConfig := &ssh.ClientConfig{
		User: host.Username,
		Auth: []ssh.AuthMethod{
			ssh.KeyboardInteractive(func(user, instruction string, questions []string, echos []bool) ([]string, error) {
				answers := make([]string, len(questions))
				for i := range answers {
					answers[i] = host.Password
				}
				return answers, nil
			}),
			ssh.Password(host.Password),
		},
		HostKeyCallback: ssh.InsecureIgnoreHostKey(),
	} // #nosec
...

ssh.KeyboardInteractive(...) simply answers with host.Password to all questions.

Please let me know if this is good enough to fix this issue, and I will submit a PR.

As per #37 (comment), we need to handle gracefully the cases where a user may open a connection already opened.