GithubHelp home page GithubHelp logo

cv_dnn's People

Contributors

ryuji0123 avatar stash-196 avatar tomoino avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

cv_dnn's Issues

settingsファイルがないと./build_docker_image.shが`/bin/sh: 1: /duser/settings/setup.sh: not found`エラー起こる

cloneしたのち

./build_docker_image.sh

を実行すると実行途中で以下のようなエラーが起こります。

Step 17/19 : RUN $SETTING_PATH/setup.sh
 ---> Running in 4c2d0d3af719
/bin/sh: 1: /duser/settings/setup.sh: not found
The command '/bin/sh -c $SETTING_PATH/setup.sh' returned a non-zero code: 127

とりあえずhttps://github.com/ryuji0123/settings をcloneして解決しましたが,ReadMeにこの手順を追加するか,settingsなしでも実行できるようにして欲しいです。

add optuna

Summary

add optuna with pytorch lightning

References

None

Notes

None

add pytest

Summary

add pytest to test functions

References

None

Notes

None

`Error response from daemon: No such container: cv_dnn "--gpus" requires API version 1.40, but the Docker daemon API version is 1.39` error while runnig `./run_docker_container.sh`

I am trying to run

./run_docker_container.sh

but it will have an error message:

Error response from daemon: No such container: cv_dnn 
"--gpus" requires API version 1.40, but the Docker daemon API version is 1.39

The whole message is as follows:

$ ./build_docker_image.sh
Sending build context to Docker daemon  398.3kB
Step 1/19 : FROM pytorch/pytorch
 ---> 349148663741
Step 2/19 : ENV DEBIAN_FRONTEND noninteractive
 ---> Using cache
 ---> 606755e73492
Step 3/19 : RUN apt-get update -qq &&           apt-get install -y                      curl libopencv-dev lsof git sudo tmux tree vim wget zsh &&              apt-get clean &&                rm -rf /var/lib/apt/lists/* &&          rm -rf /var/cache/apk/*
 ---> Using cache
 ---> 18cc2cdb7ed4
Step 4/19 : ENV WORK_PATH /workspace
 ---> Using cache
 ---> 94e70d8d5333
Step 5/19 : WORKDIR $WORK_PATH
 ---> Using cache
 ---> 0b02b62b8735
Step 6/19 : ENV PYTHONPATH "/workspace:${PYTHONPATH}"
 ---> Using cache
 ---> bd3ecf7fd142
Step 7/19 : ENV PIP_OPTIONS "--no-cache-dir --progress-bar off"
 ---> Using cache
 ---> 3c8b9432d78f
Step 8/19 : COPY docker/requirements.txt $WORK_PATH/docker/
 ---> Using cache
 ---> 2955e134c831
Step 9/19 : RUN pip install ${PIP_OPTIONS} -r $WORK_PATH/docker/requirements.txt &&             pip install ${PIP_OPTIONS} -U setuptools
 ---> Using cache
 ---> a3f06e21b091
Step 10/19 : ARG USER_ID
 ---> Using cache
 ---> a60c0c4f3a1b
Step 11/19 : ARG GROUP_ID
 ---> Using cache
 ---> 96845eff52c7
Step 12/19 : RUN addgroup --gid $GROUP_ID duser &&     adduser --disabled-password --gecos '' --uid $USER_ID --gid $GROUP_ID duser &&     adduser duser sudo &&     echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
 ---> Using cache
 ---> 2f0ebe2869b0
Step 13/19 : USER duser
 ---> Using cache
 ---> 24eea61e4120
Step 14/19 : ENV SETTING_PATH /duser/settings
 ---> Using cache
 ---> c436624f00e5
Step 15/19 : COPY docker/settings $SETTING_PATH
 ---> 672bd8def242
Step 16/19 : WORKDIR $SETTING_PATH
 ---> Running in 168161460adb
Removing intermediate container 168161460adb
 ---> 639901b9ea1f
Step 17/19 : RUN $SETTING_PATH/setup.sh
 ---> Running in 0b87df537d2a
Install to "/home/duser/.vim/dein/repos/github.com/Shougo/dein.vim"...

git is /usr/bin/git

Begin fetching dein...
Cloning into '/home/duser/.vim/dein/repos/github.com/Shougo/dein.vim'...
Done.

Please add the following settings for dein to the top of your vimrc (Vim) or init.vim (NeoVim) file:


"dein Scripts-----------------------------
if &compatible
  set nocompatible               " Be iMproved
endif

" Required:
set runtimepath+=/home/duser/.vim/dein/repos/github.com/Shougo/dein.vim

" Required:
if dein#load_state('/home/duser/.vim/dein')
  call dein#begin('/home/duser/.vim/dein')

  " Let dein manage dein
  " Required:
  call dein#add('/home/duser/.vim/dein/repos/github.com/Shougo/dein.vim')

  " Add or remove your plugins here like this:
  "call dein#add('Shougo/neosnippet.vim')
  "call dein#add('Shougo/neosnippet-snippets')

  " Required:
  call dein#end()
  call dein#save_state()
endif

" Required:
filetype plugin indent on
syntax enable

" If you want to install not installed plugins on startup.
"if dein#check_install()
"  call dein#install()
"endif

"End dein Scripts-------------------------


Done.
Complete setup dein!
Removing intermediate container 0b87df537d2a
 ---> f8a8db256c81
Step 18/19 : SHELL ["/bin/zsh", "-c"]
 ---> Running in 56fb03db74b3
Removing intermediate container 56fb03db74b3
 ---> d43c9c421bec
Step 19/19 : WORKDIR $WORK_PATH
 ---> Running in 63c954568f67
Removing intermediate container 63c954568f67
 ---> f29ebf535e82
Successfully built f29ebf535e82
Successfully tagged cv_dnn:latest
ryoto@gorgon:~/cv_dnn$ ./run_docker_container.sh
Error response from daemon: No such container: cv_dnn
"--gpus" requires API version 1.40, but the Docker daemon API version is 1.39
ryoto@gorgon:~/cv_dnn$ 

My GPU is
GPU: Tesla K80, CUDA10.0, cuDNNv7.5.0

add flake8

Summary

add flake8 to check coding style

References

None

Notes

None

add template files

Summary

Add template files and write some instructions on how to use them.

References

None

Notes

None

replace yacs with hydra

Summary

Replace yacs with hydra. You can see skew_mixup. Basically, you can copy and paste them. Also, add multiple datasets and models in skew_mixup such as cifar100 and ResNet18.

References

hydra-mlflow-optuna

Notes

  1. For a while, we don't support distributed training with pytorch-lightning. You can delete codes to use distributed training like:
(train.py)
trainer = pl.Trainer(
        callbacks=[checkpoint_callback],
        distributed_backend=args.TRAIN.DISTRIBUTED_BACKEND, # unnecessary
        gpus=args.TRAIN.GPUS, # change to 1
        logger=mlflow_logger,
        max_epochs=args.TRAIN.MAX_EPOCHS,
        replace_sampler_ddp=False, # unnecessary
    )

(config/defaults.py)
_C.TRAIN.DISTRIBUTED_BACKEND = 'ddp'  # unnecessary
_C.TRAIN.GPUS = 2 # unnecessary
  1. What you should do by yourself is to write configuration for optuna with hydra. You can see references to do it.
  2. In next week, you'll modify shell and python scripts for training. Currently, we save logs and errors by ourselves but we should do it with hydra. In this week, you can just delete these codes and add simple shell scripts like:
#!/bin/sh
TIMESTAMP=`date +%Y-%m-%d_%H-%M-%S`
TMP_RESULTS_DIR="$(pwd)/.tmp_results/${TIMESTAMP}"
TRAIN_LOG_FILE="${TMP_RESULTS_DIR}/log.txt"
mkdir -p $TMP_RESULTS_DIR
export CUDA_VISIBLE_DEVICES=0
nohup python -u train.py \
  -m data=cifar10,cifar100 model=alexnet,resnet18 \
  > $TRAIN_LOG_FILE &
sleep 1s
tail -f $TRAIN_LOG_FILE
    finally:
        run_id = mlflow_logger.run_id
        if run_id is not None:
            with open(args_file_path, 'w') as f: # unnecessary
                with redirect_stdout(f):
                    print(args.dump())

            mlflow_client = MlflowClient()
            mlflow_client.log_artifact(run_id, args_file_path) # unnecessary
            mlflow_client.log_artifact(run_id, train_log_file_path) # unnecessary
            if exist_error:
                mlflow_client.log_artifact(run_id, error_file_path)
            rmtree(tmp_results_dir, ignore_errors=True) # maybe unnecessary

Of course, if you know how to save logs and config files generated by hydra in mlflow and delete them, it's nice to do it in this week.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.