GithubHelp home page GithubHelp logo

teltec / everest Goto Github PK

View Code? Open in Web Editor NEW
7.0 7.0 3.0 1.24 MB

Do incremental backups and store them on Amazon S3 (NOT READY FOR PRODUCTION YET)

License: Mozilla Public License 2.0

C# 99.48% XSLT 0.44% Batchfile 0.08%
aws-s3 backup backup-files incremental restore-plans windows

everest's People

Contributors

jweyrich avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

everest's Issues

S3 Bucket Lifecycle Rule is not auto-configured

During the creation of an S3 bucket, the user is allowed to chose between storage classes, however, if the user selects IA (infrequent access), a Lifecycle rule should be created, but it's currently not.

SqlDateTime overflow during versioning

See attached log.

PlanExecutor\Versioning\IncrementalFileVersioner.cs:593:Save - BEGIN STEP 2
2015-09-17 13:47:06.4455 ERROR BUG: Failed to insert/update BackupPlanFile => {"Id":null,"BackupPlan":{},"StorageAccountType":1,"StorageAccount":{},"Path":"Z:\Empresa\Gerencia Tecnica\Service Desk\Service Desk\Teltec as a Service\Relatórios\NGFWAAS~$ocedimento Relatório WatchGuard.docx","LastSize":0,"LastWrittenAt":"0001-01-01T00:00:00","LastChecksum":null,"LastStatus":1,"PreviousLastStatus":1,"CreatedAt":"2015-09-17T14:08:58.7765746Z","UpdatedAt":"2015-09-17T14:12:17.6027032Z","PathNode":{},"Versions":[],"IsSaved":false} System.Data.SqlTypes.SqlTypeException: SqlDateTime overflow. Must be between 1/1/1753 12:00:00 AM and 12/31/9999 11:59:59 PM.

2015-09-17.log

Filenames containing some unicode sequences aren't handled correctly

The backup operation doesn't work properly with filenames that contain some unicode sequences (combining characters). For example, , , etc.
Currently, files containing these characters are not processed by the backup operation.

Reported by Raissa Marcon Constante.

PathTooLongException

When handling files that have a path length larger than 260 characters (Windows' MAX_PATH), the following exception is thrown:

[PathTooLongException]: The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be less than 248 characters.

The .NET API does not support these long paths because of many reasons - See Long Paths in .NET - While the Windows API supports them, I believe it's nearly impossible to get them to work seamlessly and transparently. We could opt for the \\?\ prefix but it will have other impacts. Quoting the previously referenced article:

First about security, the ?\ prefix not only enables long paths; it causes the path to be passed to the file system with minimal modification by the Windows APIs. A consequence is that ?\ turns off file name normalization performed by Windows APIs, including removing trailing spaces, expanding ‘.’ and ‘..’, converting relative paths into full paths, and so on. The existence of FileIOPermissions in .NET means that we absolutely have to work with normalized paths, or risk exposing a security threat. So we knew that if we wanted to use the ?\ prefix as part of the long path solution, we’d need the ability to normalize these paths as expected.

Another concern is inconsistent behavior that would result by exposing long path support. Long paths with the ?\ prefix can be used in most of the file-related Windows APIs, but not all Windows APIs. For example, LoadLibrary, which maps a module into the address of the calling process, fails if the file name is longer than MAX_PATH. So this means MoveFile will let you move a DLL to a location such that its path is longer than 260 characters, but when you try to load the DLL, it would fail. There are similar examples throughout the Windows APIs; some workarounds exist, but they are on a case-by-case basis.

Another factor, which is considered more of a pain factor, is compatibility with other Windows-based applications and the Windows shell itself, which only work with paths shorter than MAX_PATH (note that the Vista shell attempts to soften this limit, briefly described below). This means that if .NET supports long paths, then we’d let you create files that you couldn’t access via Explorer or the command prompt.

Official documentation can be found in Naming Files, Paths, and Namespaces.

Editing a plan and changing its storage account won't work properly

What happens if the user edits a BackupPlan or RestorePlan and changes its storage account?

Currently, the application would need to update all entities that reference the previous stored account (BackupPlan, RestorePlan, BackupPlanFile, RestorePlanFile, BackupedFile, RestoredFile, etc), and it would take dozens of minutes if there are thousands of rows in those tables.

A possible and simple solution would be to create a new join-table (id, plan_id, account_id) to associate plans to storage accounts, and change all references from storage_account_id to the_join_table_id. Doing so will make all entities reference the storage account indirectly, avoiding the need to update them all when the storage account is changed in the plan.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.