GithubHelp home page GithubHelp logo

facebookarchive / conceal Goto Github PK

View Code? Open in Web Editor NEW
3.0K 176.0 432.0 42.96 MB

Conceal provides easy Android APIs for performing fast encryption and authentication of data.

Home Page: http://facebook.github.io/conceal/

License: Other

Shell 0.43% Java 35.93% Makefile 0.44% C 1.55% IDL 0.38% Python 1.53% C++ 59.69% Prolog 0.05%

conceal's Introduction

What is Conceal? Build Status

Conceal provides a set of Java APIs to perform cryptography on Android. It was designed to be able to encrypt large files on disk in a fast and memory efficient manner.

The major target for this project is typical Android devices which run old Android versions, have low memory and slower processors.

Unlike other libraries, which provide a Smorgasbord of encryption algorithms and options, Conceal prefers to abstract this choice and use sane defaults. Thus Conceal is not a general purpose crypto library, however it aims to provide useful functionality.

Upgrading version? Check the Upgrade notes for key compatibility!

IMPORTANT: Initializing the library loader

Since v2.0.+ (2017-06-27) you will need to initialize the native library loader. This step is needed because the library loader uses the context. The highly suggested way to do it is in the application class onCreate method like this:

import com.facebook.soloader.SoLoader;
public class MyApplication extends Application {
    @Override
    public void onCreate() {
        super.onCreate();
        SoLoader.init(this, false);
    }
}

Quick start

Setup options

  1. Use Maven Central: Available on maven central under com.facebook.conceal:conceal:2.0.1@aar as an AAR package. If you use Android Studio and select the library using the UI, make sure to change build.gradle to include the @aar suffix. Otherwise the library won't be included.

  2. Build using gradle

./gradlew build

It uses gradlew so it takes care of downloading Gradle and all the dependencies it needs. Output will be in /build/outputs/aar/ directory.

  1. Use prebuilt binaries: http://facebook.github.io/conceal/documentation/. (linked documentation needs update)
An aside on KitKat

Conceal predates Jellybean 4.3. On KitKat, Android changed the provider for cryptographic algorithms to OpenSSL. The default Cipher stream however still does not perform well. When replaced with our Cipher stream (see BetterCipherInputStream), the default implementation is competitive against Conceal. On older phones, Conceal is faster than the system provided libraries.

Re-build OpenSSL library

You can run make from the openssl directory. It will download the code and copile the libraries for each architecture.

# go to /third-party/openssl
make

Before running any test!

Test uses BUCK build tool. BUCK uses the source code for OpenSSL. If you didn't already rebuilt OpenSSL form scrach (previous item) then run this:

# go to /third-party/openssl
make clone

That will download the OpenSSL code to a subdirectory.

Running unit tests

# C++ tests
buck test :cpp

Running integration tests

# Emulator/device tests
./instrumentTest/crypto/run

Since Conceal uses native libraries, the only way to run a test on the entire encryption process is using integration tests.

Running Benchmarks

./benchmarks/run \
  benchmarks/src/com/facebook/crypto/benchmarks/CipherReadBenchmark.java \
  -- -Dsize=102400

This script runs vogar with caliper benchmarks. You can also specify all the options caliper provides.

Usage

Entity and keys

Entity: this is a not-secret identifier of your data. It's used for integrity check purposes (to know that the content has not been tampered) and also to verify it was not swapped with another valid encrypted content/file.

Key: the key is provided by the KeyChain implementation passed to the Crypto object. So each time a new encryption is requested, the key is requested to the KeyChain. The key is generated randomly the first time on demand. You might change the implementation by we strongly suggest to generate a random value. If the encryption key needs for some reason to be based on a text password, you can try using the PasswordBasedKeyGenerator object.

Encryption

// Creates a new Crypto object with default implementations of a key chain
KeyChain keyChain = new SharedPrefsBackedKeyChain(context, CryptoConfig.KEY_256);
Crypto crypto = AndroidConceal.get().createDefaultCrypto(keyChain);

// Check for whether the crypto functionality is available
// This might fail if Android does not load libaries correctly.
if (!crypto.isAvailable()) {
  return;
}

OutputStream fileStream = new BufferedOutputStream(
  new FileOutputStream(file));

// Creates an output stream which encrypts the data as
// it is written to it and writes it out to the file.
OutputStream outputStream = crypto.getCipherOutputStream(
  fileStream,
  Entity.create("entity_id"));

// Write plaintext to it.
outputStream.write(plainText);
outputStream.close();

Decryption

// Get the file to which ciphertext has been written.
FileInputStream fileStream = new FileInputStream(file);

// Creates an input stream which decrypts the data as
// it is read from it.
InputStream inputStream = crypto.getCipherInputStream(
  fileStream,
  Entity.create("entity_id"));

// Read into a byte array.
int read;
byte[] buffer = new byte[1024];

// You must read the entire stream to completion.
// The verification is done at the end of the stream.
// Thus not reading till the end of the stream will cause
// a security bug. For safety, you should not
// use any of the data until it's been fully read or throw
// away the data if an exception occurs.
while ((read = inputStream.read(buffer)) != -1) {
  out.write(buffer, 0, read);
}

inputStream.close();

If you don't have a lot of data to encrypt, you could use the convenience functions:

byte[] cipherText = crypto.encrypt(plainText, Entity.create("mytext"));

byte[] plainText = crypto.decrypt(cipherText, Entity.create("mytext"));

Integrity

OutputStream outputStream = crypto.getMacOutputStream(fileStream, entity);
outputStream.write(plainTextBytes);
outputStream.close();

InputStream inputStream = crypto.getMacInputStream(fileStream, entity);

// Will throw an exception if mac verification fails.
// You must read the entire stream to completion.
// The verification is done at the end of the stream.
// Thus not reading till the end of the stream will cause
// a security bug. For safety, you should not
// use any of the data until it's been fully read or throw
// away the data if an exception occurs.
while((read = inputStream.read(buffer)) != -1) {
  out.write(buffer, 0, read);
}
inputStream.close();

Upgrade notes

Starting with v1.1 recommended encryption will use a 256-bit key (instead of 128-bit). This means stronger security. You should use this default.

If you need to read from an existing file, you still will need 128-bit encryption. You can use the old way of creating Crypto objects as it preserves its 128-bit behavior. Although ideally you should re-encrypt that content with a 256-bit key.

Also there's an improved way of creating Entity object which is platform independent. It's strongly recommended for new encrypted items although you need to stick to the old way for already encrypted content.

Existing code still with 128-bit keys and old Entity (deprecated)

// this constructor creates a key chain that produces 128-bit keys
KeyChain keyChain = new SharedPrefsBackedKeyChain(context);
// this constructor creates a crypto that uses  128-bit keys
Crypto crypto = new Crypto(keyChain, library);
Entity entity = new Entity(someStringId);

New code using 256-keys and Entity.create

We recommend the use of the factory class AndroidConceal.

// explicitely create 256-bit key chain
KeyChain keyChain = new SharedPrefsBackedKeyChain(context, CryptoConfig.KEY_256);
// create the default crypto (expects 256-bit key)
AndroidConceal.get().createDefaultCrypto(keyChain);
// factory class also has explicit methods: createCrypto128Bits and ceateCrypto256Bits if desired.
Entity entity = Entity.create(someStringId);

Troubleshooting

I'm getting NoSuchFieldError on runtime

If you hit an error on runtime and it says something similar to:

java.lang.NoSuchFieldError: no field with name='mCtxPtr' signature='J' in class Lcom/facebook/crypto/cipher/NativeGCMCipher;

This happens because native code needs to refer to Java fields/methods. For doing so it uses typical JNI functions which receive the name and signature. At the same time tools like proguard trim off or rename class members in order to get smaller executables. Normally this process is run on release versions. When native code request the member, it's not present anymore.

To avoid this kind of problems exceptions can be defined. You will need to configure proguard with the rules defined in proguard_annotations.pro. You can use the file as is, or you can include its content in your own proguard configuration file.

conceal's People

Contributors

bailan avatar christianroman avatar corymsmith avatar erdemolkun avatar flarnie avatar helios175 avatar matthewmichihara avatar mburman avatar orhanobut avatar pitel avatar siyengar avatar y-yagi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

conceal's Issues

ClassNotFoundException Native Crash On Lollipop using Gradle/AAR

Trying to execute:
new Crypto(new SharedPrefsBackedKeyChain(context), new SystemNativeCryptoLibrary());

Results in:
A/art﹕ art/runtime/check_jni.cc:65] JNI DETECTED ERROR IN APPLICATION: JNI GetFieldID called with pending exception 'java.lang.ClassNotFoundException' thrown in unknown throw location
A/art﹕ art/runtime/check_jni.cc:65] in call to GetFieldID
A/art﹕ art/runtime/check_jni.cc:65] from java.lang.String java.lang.Runtime.nativeLoad(java.lang.String, java.lang.ClassLoader, java.lang.String)

crypto.jar missing

According to http://facebook.github.io/conceal/documentation/ I should:

"Add a dependency to crypto.jar from java code using your favorite dependency management system, and drop the .so files in libs.zip into your libs/ folder."

Except crypto.jar has been removed, and replaced with conceal_android.jar and libconceal.jar.

This was done in SVN revision 70:

Revision: 70
Author: subodh.iyengar
Date: 10 February 2015 12:34:56 PM
Message:

update artefacts

Added : /branches/gh-pages/downloads/conceal_android.jar
Deleted : /branches/gh-pages/downloads/crypto.jar
Added : /branches/gh-pages/downloads/libconceal.jar
Modified : /branches/gh-pages/downloads/libs.zip

Add APIs to make NativeMacLayeredInputStream more secure

The MAC streams assume that the MAC is supplied after the end of stream's data and the input stream returns all the data, and checks the MAC only after EOF is read. Thus, one needs to treat data read from MacInputStream as insecure until the stream is read to EOF. The documentation for Crypto.getMacInputStream() says only that you need to read all of the stream to avoid security vulnerabilities, but doesn't say that you need to treat all data read from the stream as untrusted until you've read all of the data from the stream.

I also posit that there are very few cases when doing something other than reading all of such stream into a buffer and doing anything with it only after the stream has verified the MAC is secure; thus I'd consider it more prudent if the stream were to return only data that have already been verified. (That'd require changing the wire format produced to use a notion of a block or always reading the whole stream into memory at once.)

Conceal Help

I'm currently working on a project and i've been tasked to make sure the contents/data inside the apk is secure from been accessed by users, i'm been research for a couple of days now and i finally find this library, please do you think this will provide solution for this task (Encrypting the data, so user will not have access to it), if not? is there any alternative...? (i'll appreciate your suggestion). lastly. To encrypt data with this library? data must be on SD card or inside the internal memory?

error on com.facebook.crypto.cipher.NativeGCMCipherException: decryptFinal

Hi,

I'm using this library but when I call the decrypt method I get this error:

09-02 08:30:40.549: W/System.err(9205): com.facebook.crypto.cipher.NativeGCMCipherException: decryptFinal
09-02 08:30:40.549: W/System.err(9205):     at com.facebook.crypto.cipher.NativeGCMCipher.decryptFinal(NativeGCMCipher.java:108)
09-02 08:30:40.549: W/System.err(9205):     at com.facebook.crypto.streams.NativeGCMCipherInputStream.ensureTagValid(NativeGCMCipherInputStream.java:126)
09-02 08:30:40.549: W/System.err(9205):     at com.facebook.crypto.streams.NativeGCMCipherInputStream.read(NativeGCMCipherInputStream.java:91)
09-02 08:30:40.549: W/System.err(9205):     at com.facebook.crypto.streams.NativeGCMCipherInputStream.read(NativeGCMCipherInputStream.java:76)
09-02 08:30:40.549: W/System.err(9205):     at com.facebook.crypto.Crypto.decrypt(Crypto.java:131)
09-02 08:30:40.549: W/System.err(9205):     at com.example.sampleconceal.MainActivity.onCreate(MainActivity.java:46)
09-02 08:30:40.549: W/System.err(9205):     at android.app.Activity.performCreate(Activity.java:5008)
09-02 08:30:40.549: W/System.err(9205):     at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1079)
09-02 08:30:40.549: W/System.err(9205):     at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2023)
09-02 08:30:40.549: W/System.err(9205):     at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2084)
09-02 08:30:40.549: W/System.err(9205):     at android.app.ActivityThread.access$600(ActivityThread.java:130)
09-02 08:30:40.549: W/System.err(9205):     at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1195)
09-02 08:30:40.549: W/System.err(9205):     at android.os.Handler.dispatchMessage(Handler.java:99)
09-02 08:30:40.549: W/System.err(9205):     at android.os.Looper.loop(Looper.java:137)
09-02 08:30:40.549: W/System.err(9205):     at android.app.ActivityThread.main(ActivityThread.java:4745)
09-02 08:30:40.549: W/System.err(9205):     at java.lang.reflect.Method.invokeNative(Native Method)
09-02 08:30:40.549: W/System.err(9205):     at java.lang.reflect.Method.invoke(Method.java:511)
09-02 08:30:40.549: W/System.err(9205):     at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:786)
09-02 08:30:40.549: W/System.err(9205):     at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:553)
09-02 08:30:40.549: W/System.err(9205):     at dalvik.system.NativeStart.main(Native Method)

I'm using a Genymotion emulator to run my sample app.

Performance Issues

I'm seeing performance around 2.5 seconds to encrypt 150k on a Nexus 5 running Lollipop. Is this typical?

Increment version number when you make changes

Your current deployment process to Maven Central appears to be seriously flawed. Since you keep re-releasing as version 1.0.0, it breaks Maven's caching behaviour - people with cached versions of the library won't get the updates.

I ran into this when doing a build on a new machine and getting a compile error since you've changed the package structure.

I'm not the first person to run into this - e.g. #60, #58 - and I'm sure others will too. It also likely means a lot of devs out there using Maven/Gradle aren't getting the latest version.

Gradle Import Not Importing OpenSSL Fixes?

Google is still reporting that I am referencing vulnerable OpenSSL versions. Does the documented import method actually import recent fixes? If you guys are updating OpenSSL and iterating, why isn't the version changing? Currently importing as follows:

compile 'com.facebook.conceal:conceal:1.0.0@aar'

Decrypt on Samsung N7100 sometimes fails

My logic is simple:
encrypt with crypto.encrypt(byte[], Entity), and decrypt with crypto.decrypt(byte[], Entity). It works on most of my devices. However, on my Samsung N7100, there are 50+% probabilities that decrypt will fail with exception below.

W/System.err( 9705): com.facebook.crypto.cipher.NativeGCMCipherException: decryptFinal
W/System.err( 9705): at com.facebook.crypto.cipher.NativeGCMCipher.decryptFinal(NativeGCMCipher.java:108)
W/System.err( 9705): at com.facebook.crypto.streams.NativeGCMCipherInputStream.ensureTagValid(NativeGCMCipherInputStream.java:126)
W/System.err( 9705): at com.facebook.crypto.streams.NativeGCMCipherInputStream.read(NativeGCMCipherInputStream.java:91)
W/System.err( 9705): at com.facebook.crypto.streams.NativeGCMCipherInputStream.read(NativeGCMCipherInputStream.java:76)
W/System.err( 9705): at com.facebook.crypto.Crypto.decrypt(Crypto.java:131)
W/System.err( 9705): at android.taobao.weakconceal.WeakConceal.read(WeakConceal.java:81)
W/System.err( 9705): at android.taobao.weakconceal.WeakConceal.read(WeakConceal.java:40)
W/System.err( 9705): at com.taobao.testwc.MainActivity$1.onClick(MainActivity.java:33)
W/System.err( 9705): at android.view.View.performClick(View.java:4223)
W/System.err( 9705): at android.view.View$PerformClick.run(View.java:17275)
W/System.err( 9705): at android.os.Handler.handleCallback(Handler.java:615)
E/AudioMixer( 1934): MOON > checkSEC_PCM ++ 1
W/System.err( 9705): at android.os.Handler.dispatchMessage(Handler.java:92)
W/System.err( 9705): at android.os.Looper.loop(Looper.java:137)
W/System.err( 9705): at android.app.ActivityThread.main(ActivityThread.java:4898)
W/System.err( 9705): at java.lang.reflect.Method.invokeNative(Native Method)
W/System.err( 9705): at java.lang.reflect.Method.invoke(Method.java:511)
W/System.err( 9705): at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1008)
W/System.err( 9705): at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:775)
W/System.err( 9705): at dalvik.system.NativeStart.main(Native Method)

Store the keychain in sdcard

I'd like to implement a keychain class which stores the keychain in sdcard. Most of the logic will be copied from SharedPrefsBackedKeyChain. Will this expose any potential security hole? You see that storing the keychain on sdcard means that it will be seen by other apps.

Streaming an encrypted video file to a VideoView

I have a encrypted mp4 file about 130M that I want to play in a VideoView. What is the best way to stream it? I don't want to copy to an unencrypted file and then use that is the time it takes to decrypt it is prohibitive on slower devices. What I have so far is copying to a file and start the VideoView by giving it that buffered file before the decryption finishes.


public class DecryptVideoTask extends AsyncTask<Void, File, File> {
    public static final String TAG = DecryptVideoTask.class.getCanonicalName();
    public static final int READ_AHEAD_BYTES = 65469100;

    private Context context;
    private VideoView videoView;
    private String encryptedFilePath;
    private ProgressDialog progressDialog;

    /**
     *
     * @param context currentContext
     * @param videoView VideoView that will play the video
     * @param encryptedFilePath conceal encrypted video
     * @param progressDialog progressDialog to hide when the video starts
     */
    public DecryptVideoTask(Context context, VideoView videoView, String encryptedFilePath, ProgressDialog progressDialog) {
        this.context = context;
        this.videoView = videoView;
        this.encryptedFilePath = encryptedFilePath;
        this.progressDialog = progressDialog;
    }

    @Override
    protected File doInBackground(Void... params) {
        File bufferFile = null;

        try {

            bufferFile = File.createTempFile("temp", ".mp4", context.getCacheDir());

            BufferedOutputStream bufferOS = new BufferedOutputStream(
                    new FileOutputStream(bufferFile));

            // getting CipherInputStream.  encryptedFilePath is the path to a conceal encrypted file
            InputStream is = Application.getInstance().getCryptoManager().decrypt(new File(encryptedFilePath));
            BufferedInputStream bis = new BufferedInputStream(is, 524288);

            byte[] buffer = new byte[32768];
            int numRead;
            long totalRead = 0;
            boolean started = false;

            long startTime = System.currentTimeMillis();
            while ((numRead = bis.read(buffer)) != -1) {

                bufferOS.write(buffer, 0, numRead);
                bufferOS.flush();
                totalRead += numRead;
                Log.d(TAG, "total read: " + totalRead);
                if (totalRead > READ_AHEAD_BYTES && !started) {
                    Log.d(TAG, "decryption took: " + (System.currentTimeMillis() - startTime) + "ms");
                    Log.d(TAG, "starting video: " + encryptedFilePath);
                    publishProgress(bufferFile);
                    started = true;
                }

            }

            bufferOS.close();
            bis.close();

            if (!started) {
                Log.d(TAG, "file is decrypted but video has not been started, starting now: " + encryptedFilePath);
                publishProgress(bufferFile);
                started = true;
            }
        } catch (IOException e) {
            Log.e(TAG, e.getMessage(), e);
            return bufferFile;
        }

        return bufferFile;
    }

    @Override
    protected void onProgressUpdate(File... values) {
        videoView.setVideoURI(Uri.parse("file://" + values[0].getAbsolutePath()));
        videoView.requestFocus();
        videoView.start();

        if (progressDialog != null && progressDialog.isShowing()) {
            progressDialog.dismiss();
        }
    }
}

The video starts but depending on what buffer sizes I fiddle with (READ_AHEAD_BYTES) the video may error out after a few seconds with

01-23 09:54:13.219  12087-12099/com.github.browep E/MediaPlayer﹕ error (1, -1004)
01-23 09:54:13.219  12087-12087/com.github.browep E/MediaPlayer﹕ Error (1,-1004)

and a popup that says "can't play this video". Is there a better way to do this? FileDescriptor? something?

iOS Support

Is there any chance that this library will also be available on iOS in the near future?

How to get addition length of file after encrypt?

I want to get the length of encrypted file (include the pass phrase) like this:
long expectedLength = 360932 // I get this from the server
long beforeLength = 360932; // after download file
long afterLength = 360962; // after encrypt file
Now I want to verify if it I already downloaded this file by the file size like this:

if( expectedLength  == afterLength + X){   // X is the addition length of file after encrypt
   // file already downloaded
}else{
    // should download file again because there is some error while downloading
}

// get addition length, X should be 360962 - 360932 = 30, but it always returns 94
X = new Entity(myKey).getBytes().length;

Is there something wrong? Says I have to download and check size of many files, and I need the correct way to get addition length of file after encrypt.

Add a protocol id number to the wire stream

A version number does not exist right now in the stream, so it is possible to upgrade the library, and if the wire protocol or the encryption algorithm changes, we could not decrypt the data. We need to add a ids to the stream to make sure versions are cross compatible. This would add additional storage, however I think it's worth the cost

Safe guard the Cipher key stored in Shared Preferences

Hello, Thanks for such a wonderful library at the first place.

I am planning to use the conceal library for my live app. My purpose is to store user data encrypted in the device. From your documentation, I understand the Cipher key is being stored as Shared Preferences (under crypto.xml - with content below).

-
*******************************

My issue is to keep the cipher_key secure enough. On a rooted phone the Shared Preferences is easily accessible and its easy for a malicious app or hacker to get hold of the key and decrypt my user data.

Please advise on how to store the "cipher_key" securely within the app. I would like to use this without the user consent.

My app supports android versions 4.0 and above. Please guide.

com.facebook.crypto.cipher.NativeGCMCipherException: decryptFinal

I am using Android Studio and the relevant portions of my build.gradle file looks like this:

buildscript {
    repositories {
        mavenCentral()
        flatDir {
            dirs 'aars'
        }
    }
}

dependencies {
    compile fileTree(dir: 'libs', include: ['*.jar'])
    compile 'com.facebook.conceal:conceal:1.0.0@aar'
}
        Crypto crypto = new Crypto(
                new SharedPrefsBackedKeyChain(this),
                new SystemNativeCryptoLibrary());        
       String t = "test";
        try {
            byte[] enc= crypto.encrypt(t.getBytes(), new Entity("1"));
            byte[] dec = crypto.decrypt(enc, new Entity("2"));
        } catch (Exception e) {
            e.printStackTrace();
        }

This is the following exception that gets thrown:
com.facebook.crypto.cipher.NativeGCMCipherException: decryptFinal
at com.facebook.crypto.cipher.NativeGCMCipher.decryptFinal(NativeGCMCipher.java:108)
at com.facebook.crypto.streams.NativeGCMCipherInputStream.ensureTagValid(NativeGCMCipherInputStream.java:126)
at com.facebook.crypto.streams.NativeGCMCipherInputStream.read(NativeGCMCipherInputStream.java:91)
at com.facebook.crypto.streams.NativeGCMCipherInputStream.read(NativeGCMCipherInputStream.java:76)
at com.facebook.crypto.Crypto.decrypt(Crypto.java:131)

Unreadable crypted files after upgrading conceal

Hi,

I got this error after upgrading (got latest version from maven) from an older version of conceal:
java.io.IOException: Unexpected crypto version -30

So my old saved crypted file are now unreadable!
How can I fix this problem?

Thanks.

Fabio

API Request : encrypt(byte[] plainText, byte[] passwordOrKey) //return encrypted text

Since we API's like below

public byte[] encrypt(byte[] plainText);
public byte[] decrypt(byte[] cipherText);

can we have

public byte[] encrypt(byte[] plainText, byte[] passwordOrKey);
public byte[] decrypt(byte[] cipherText, byte[] passwordOrKey);

which uses given password rather than from SharedPreference, I understand I can implement Custom KeyChain, however these API's are much simpler for the cases where encryption is based on inputs from user.

PHP Implementation

Which concrete algorithms is conceal using? I want to decrypt files crypted using conceal on serverside using php, but I'm not able to figure out how to do this. Or is there maybe an easy way to do this?

BUCK file or README.MD outdated

Because of #51, i'm trying to build master branch.
(I think master branch's openssl version already updated to 1.0.1h on 5f2492f. Am i correct?)
However after 7b09a02 merged, BUCK command(buck build :crypto) does not work.

I can build project if i move commit before 7b09a02.
But please fix BUCK file or build guide on README.MD

Using Conceal with BitmapFactory.decode

Conceal does not work with the BitmapFactory.decode family of methods. Inside decode, InputStream.skip is called, which throws NotImplementedException. Writing images to a file works great, but reading fails. As a workaround, it is possible read them as a ByteArrayInputStream, then convert the byte[] to a bitmap, but it would be much cleaner if I could just pass the cipher stream to decodeBitmap.

API Request : encrypt(String plaintText) //return encrypted text

As of now, encrypted text are written / read from file, is it possible to add API to get the encrypted text for plain text and use any other way to store, say in database or shared preference.

public String encrypt(String plaintText) //return encrypted text
public String decrtpt(String encryptedText) // return decrypted text

Caused by: java.lang.UnsatisfiedLinkError: Could't load cryptox from dalvik

I just build the project and imported to my project (TestList) and I getting this error.

Note: I've imported as a library.

05-09 13:15:14.192: W/System.err(25991): com.facebook.crypto.exception.CryptoInitializationException: java.lang.UnsatisfiedLinkError: Couldn't load cryptox from loader dalvik.system.PathClassLoader[DexPathList[[zip file "/data/app/com.example.testlist-1.apk"],nativeLibraryDirectories=[/data/app-lib/com.example.testlist-1, /vendor/lib, /system/lib]]]: findLibrary returned null
05-09 13:15:14.192: W/System.err(25991):    at com.facebook.crypto.util.SystemNativeCryptoLibrary.ensureCryptoLoaded(SystemNativeCryptoLibrary.java:41)
05-09 13:15:14.192: W/System.err(25991):    at com.facebook.crypto.cipher.NativeGCMCipher.encryptInit(NativeGCMCipher.java:54)
05-09 13:15:14.192: W/System.err(25991):    at com.facebook.crypto.CipherHelper.getCipherOutputStream(CipherHelper.java:38)
05-09 13:15:14.192: W/System.err(25991):    at com.facebook.crypto.Crypto.encrypt(Crypto.java:102)
05-09 13:15:14.192: W/System.err(25991):    at com.example.testlist.MainActivity.onCreate(MainActivity.java:53)
05-09 13:15:14.192: W/System.err(25991):    at android.app.Activity.performCreate(Activity.java:5133)
05-09 13:15:14.192: W/System.err(25991):    at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1087)
05-09 13:15:14.192: W/System.err(25991):    at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2175)
05-09 13:15:14.199: W/System.err(25991):    at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2261)
05-09 13:15:14.199: W/System.err(25991):    at android.app.ActivityThread.access$600(ActivityThread.java:141)
05-09 13:15:14.199: W/System.err(25991):    at     android.app.ActivityThread$H.handleMessage(ActivityThread.java:1256)
05-09 13:15:14.199: W/System.err(25991):    at android.os.Handler.dispatchMessage(Handler.java:99)
05-09 13:15:14.199: W/System.err(25991):    at android.os.Looper.loop(Looper.java:137)
05-09 13:15:14.199: W/System.err(25991):    at android.app.ActivityThread.main(ActivityThread.java:5103)
05-09 13:15:14.199: W/System.err(25991):    at java.lang.reflect.Method.invokeNative(Native Method)
05-09 13:15:14.199: W/System.err(25991):    at java.lang.reflect.Method.invoke(Method.java:525)
05-09 13:15:14.199: W/System.err(25991):    at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:737)
05-09 13:15:14.199: W/System.err(25991):    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:553)
05-09 13:15:14.199: W/System.err(25991):    at dalvik.system.NativeStart.main(Native Method)
05-09 13:15:14.199: W/System.err(25991): Caused by: java.lang.UnsatisfiedLinkError: Couldn't load cryptox from loader dalvik.system.PathClassLoader[DexPathList[[zip file "/data/app/com.example.testlist-1.apk"],nativeLibraryDirectories=[/data/app-lib/com.example.testlist-1, /vendor/lib, /system/lib]]]: findLibrary returned null

Problems with reading from the NativeGCMCypherInputStream

EDIT: There is an update on this issue at the very bottom

Hi, I'm having issues with reading decrypted data from conceal. It looks like I can't correctly finish streaming.
I pretend there is some issue with conceal, because of when I switch my proxyStream (just the encryption part) to not run it through conceal, everything works as expected. I'm also assuming that writing is ok, there is no exception whatsoever and I can find the encrypted file on disk.

I'm proxying my data through contentprovider to allow other apps read decrypted data when the user wants it. (sharing,...)

In my content provider I'm using the openFile method to allow contentResolvers read the data

@Override
    public ParcelFileDescriptor openFile(Uri uri, String mode) throws FileNotFoundException {

        try {
            ParcelFileDescriptor[] pipe = ParcelFileDescriptor.createPipe();
            String name = uri.getLastPathSegment();
            File file = new File(name);
            InputStream fileContents = mStorageProxy.getDecryptInputStream(file);
            ParcelFileDescriptor.AutoCloseOutputStream stream = new ParcelFileDescriptor.AutoCloseOutputStream(pipe[1]);
            PipeThread pipeThread = new PipeThread(fileContents, stream);
            pipeThread.start();
            return pipe[0];
        } catch (IOException e) {
            e.printStackTrace();
        }
        return null;
    }

I guess in the Facebook app Facebook android team could be rather using a standard query() method with a byte array sent in MediaStore.MediaColumns() which is not suitable for me because of I'm not only encryptin media files and I also like the approach of streams better.

This is how I'm reading from the Inpustream. It's basically a pipe between two parcelFileDescriptors. The inputstream comes from conceal and it is a FileInputstream wrapped into a BufferedInputStream originaly.

static class PipeThread extends Thread {
        InputStream input;
        OutputStream out;

        PipeThread(InputStream inputStream, OutputStream out) {
            this.input=inputStream;
            this.out=out;
        }

        @Override
        public void run() {
                byte[] buf=new byte[1024];
                int len;

                try {
                    while ((len=input.read(buf)) > 0) {
                        out.write(buf, 0, len);
                    }

                    input.close();
                    out.flush();
                    out.close();
                }
            catch (IOException e) {
                Log.e(getClass().getSimpleName(),
                    "Exception transferring file", e);
            }
        }
    }

I've tried other methods how to read the stream, so it really shouldn't be the issue.

Finally here's the exception I'm constantly ending up with. Do you know what could be the issue?

Exception transferring file
    com.facebook.crypto.cipher.NativeGCMCipherException: decryptFinal
            at com.facebook.crypto.cipher.NativeGCMCipher.decryptFinal(NativeGCMCipher.java:108)
            at com.facebook.crypto.streams.NativeGCMCipherInputStream.ensureTagValid(NativeGCMCipherInputStream.java:126)
            at com.facebook.crypto.streams.NativeGCMCipherInputStream.read(NativeGCMCipherInputStream.java:91)
            at com.facebook.crypto.streams.NativeGCMCipherInputStream.read(NativeGCMCipherInputStream.java:76)

EDIT:
It looks like the stream is working ok, but what fails is the last iteration of reading from it. As I'm using buffer it seems like the fact that the buffer is bigger then the amount of remaiming data is causing the issue. I've been looking into sources of conceal and it seems to be ok from this regard there. Couldn't it be failing somewhere in the native layer?
Note: I've managed to get the decrypted file except its final chunk of bytes..So I have for example an incomplete image file (with last few thousands of pixels not being displayed)

Different AES options

As told here: http://stackoverflow.com/questions/22662332/heap-corruption-segv-maperr-in-android-native-code?noredirect=1#comment34648412_22662332

I am opening a issue to say that some other encoding format sould be supported, in my case AES CBC with different keysizes are a must so I needed to rewrite almost the whole library.

Aslo as it is now it should be optional to use the keyring as maybe the keys are shared between devices.
Another design decission that I didn't liked is that crypto and nativeLibrary instances must be shared within the app or recreated every time (this tries to load jni libraries every time so it is a lot more time consuming), simple singletons will be a lot more efficient.

Aside for that I found that BetterCipherInputStream does not support any cipher that requires calling doFinal (usually padded ciphers) so a major rewite was needed there (https://github.com/frisco82/conceal/blob/master/java/com/facebook/crypto/streams/BetterCipherInputStream.java), having an option to define the buffer size is a must also, depending on the use cases 256bytes it is too low and the overhead of jni will add up.

In my case I found that openssl cipherUpdate could overflow the output buffer (or maybe it was jni ReleaseByteArrayElements after a number of calls within the same context, I was unable to fully debug it so I added an extra block to the buffer to avoid the memory corruption.

Aside from that there are minor optimizations that I made in my work:

  • Key and iv does not need to be stored with the context, it makes it's own copies, so a struct is not needed, just store a pointer to the context.
  • NativeFaliure should be cached java-side as those are more jni calls for no reason.
  • Init method can be changed to support multiple key sizes easily and using CipherInit the code will be cleaner.

TailInputStream.read() and NativeMacLayeredInputStream.read() weirdness

TailInputStream.read() reads the underlying InputStream one byte at a time, but completely ignores the value of the read byte. Instead, it returns -1 if end of stream has been reached, or 1 if one byte was read. This breaks the contract of InputStream.read() which TailInputStream.read() overrides.

NativeMacLayeredInputStream.read() reads the underlying delegate InputStream one byte at a time using InputStream.read(). If the underlying InputStream returns a byte with value 0, NativeMacLayeredInputStream.read() skips adding the byte into the MAC. At the moment this issue is compounded by the fact that the delegate InputStream is actually a TailInputStream whose read() doesn't even return the actual value of the byte read.

I have not tested whether this is true -- I've only looked at the source code. See https://github.com/facebook/conceal/blob/master/java/com/facebook/crypto/streams/TailInputStream.java#L37 and https://github.com/facebook/conceal/blob/master/java/com/facebook/crypto/streams/NativeMacLayeredInputStream.java#L69.

Random number generated exception

I'm getting this exception thrown:
https://github.com/facebook/conceal/blob/master/java/com/facebook/crypto/keychain/SecureRandomFix.java#L69

On both Robolectric and on a Motorola Solutions MC40 Android device running Android 4.1.1. The code works on an HTC One Max, however.

It seems like an error to me as well to hide the causing exception (i.e. t is not used here, it should be set as the cause on the exception being thrown), since I can't even tell what is broken until I break out a debugger or download and modify the source. Unfortunately I can't even drop in my own version of the fix in without modifying source since the use of this class is private:
https://github.com/facebook/conceal/blob/24a97da5aa8bd2ac497acfb878d16dbe8f669fa8/java/com/facebook/crypto/keychain/SharedPrefsBackedKeyChain.java#L52

Robolectric, for example, just doesn't need this.

Latest version on maven central have an unsafe version of OpenSSL

Im using gradle to import conceal.
compile 'com.facebook.conceal:conceal:1.0.0@aar'

Google now complains that im using openSSL 1.0.1e, which is unsafe.
This was upgraded for conceal in july:
7d0ca93

But maven central repo has not been updated since may: http://search.maven.org/#search%7Cga%7C1%7Cconceal

When i run unzip -p YourApp.apk | strings | grep "OpenSSL", which google tells me to do.
It finds alot of OpenSSL, 1.0.1e strings in my apk package, which was the version coneal used before the upgrade in july.
Iv gone through the other libraries which are, dagger, otto and gson, and I believe they dont use openSSL.

So is there anything i can do to get conceal updated with the latest version through gradle?

why entity is used?

Hi,
I would like to know,why entity is used.Because while encrypt i used entity as test and decrypt as 'test1' but got correct output.Please let me know why that is used?
Thanks in Advance

Encrypt message on one device and decrypt on another

Hi,

I have a strange problem with sharing cipher message between two devices. I make a chat which is using this cipher to encode share message, image, video and voice. Everything works well when i type to myself, that's mean encode and decode on same device.

When I write message to someone else e.g.
message = "Hi buddy!";
key = "3oqpnnnrhip5gdh02mt6bnslke";
message = String.valueOf(Base64.encodeToString(crypto.encrypt(Base64.encode(message[0].getBytes(), Base64.DEFAULT), new Entity(key)), Base64.DEFAULT));
after encode: "AQEt6KNevPCjsgllKD985iX1lqpwYyzObkYclEK4i/fxxp0bjwLJzIcNAg=="

on another device i get the same encode message and key. But when i try decode i always get IOException.

"com.facebook.crypto.cipher.NativeGCMCipherException: decryptFinal
12-26 09:37:16.399 2327-2327/com.incotalk W/System.err﹕ at com.facebook.crypto.cipher.NativeGCMCipher.decryptFinal(NativeGCMCipher.java:108)
12-26 09:37:16.399 2327-2327/com.incotalk W/System.err﹕ at com.facebook.crypto.streams.NativeGCMCipherInputStream.ensureTagValid(NativeGCMCipherInputStream.java:126)
12-26 09:37:16.399 2327-2327/com.incotalk W/System.err﹕ at com.facebook.crypto.streams.NativeGCMCipherInputStream.read(NativeGCMCipherInputStream.java:91)
12-26 09:37:16.399 2327-2327/com.incotalk W/System.err﹕ at com.facebook.crypto.streams.NativeGCMCipherInputStream.read(NativeGCMCipherInputStream.java:76)
12-26 09:37:16.399 2327-2327/com.incotalk W/System.err﹕ at com.facebook.crypto.Crypto.decrypt(Crypto.java:131)
12-26 09:37:16.399 2327-2327/com.incotalk W/System.err﹕ at com.incotalk.activity_talk.decode(activity_talk.java:1121)
12-26 09:37:16.399 2327-2327/com.incotalk W/System.err﹕ at com.incotalk.activity_talk$6.onItemClick(activity_talk.java:446)
12-26 09:37:16.399 2327-2327/com.incotalk W/System.err﹕ at android.widget.AdapterView.performItemClick(AdapterView.java:298)
12-26 09:37:16.399 2327-2327/com.incotalk W/System.err﹕ at android.widget.AbsListView.performItemClick(AbsListView.java:1086)
12-26 09:37:16.399 2327-2327/com.incotalk W/System.err﹕ at android.widget.AbsListView$PerformClick.run(AbsListView.java:2855)
12-26 09:37:16.399 2327-2327/com.incotalk W/System.err﹕ at android.widget.AbsListView$1.run(AbsListView.java:3529)
12-26 09:37:16.399 2327-2327/com.incotalk W/System.err﹕ at android.os.Handler.handleCallback(Handler.java:615)
12-26 09:37:16.399 2327-2327/com.incotalk W/System.err﹕ at android.os.Handler.dispatchMessage(Handler.java:92)
12-26 09:37:16.399 2327-2327/com.incotalk W/System.err﹕ at android.os.Looper.loop(Looper.java:137)
12-26 09:37:16.399 2327-2327/com.incotalk W/System.err﹕ at android.app.ActivityThread.main(ActivityThread.java:4745)
12-26 09:37:16.399 2327-2327/com.incotalk W/System.err﹕ at java.lang.reflect.Method.invokeNative(Native Method)
12-26 09:37:16.399 2327-2327/com.incotalk W/System.err﹕ at java.lang.reflect.Method.invoke(Method.java:511)
12-26 09:37:16.399 2327-2327/com.incotalk W/System.err﹕ at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:786)
12-26 09:37:16.399 2327-2327/com.incotalk W/System.err﹕ at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:553)
12-26 09:37:16.399 2327-2327/com.incotalk W/System.err﹕ at dalvik.system.NativeStart.main(Native Method)"

I decode with this line of code:
byte[] messageB = Base64.decode(message, Base64.DEFAULT);
return new String(
Base64.decode(crypto.decrypt(messageB, new Entity(attachmentKeyDecoded)),Base64.DEFAULT), "UTF-8");

return new String(
Base64.decode(crypto.decrypt(messageB, new Entity(attachmentKeyDecoded)), Base64.DEFAULT),
"UTF-8"
)

Please explain why it doesn't work.

Fix packaging of maven artifact

If you take a look at the .pom file in maven central, you'll notice that the packaging is 'aar.asc'.

http://search.maven.org/#artifactdetails%7Ccom.facebook.conceal%7Cconceal%7C1.0.0%7Caar.asc

But it should be only 'aar'. I'm not sure what's causing this issue in your build.gradle file. You are using old build tools, that might be the reason.

To fix the dependency in gradle I used: compile 'com.facebook.conceal:conceal:1.0.0@aar'

It works, but isn't that pretty. Otherwise the maven artifact works great!

Replace the RNG based IV with deterministic construction

Currently we use an PRNG based IV AES-GCM. NIST says that this limits the use of authentication function to 2^32 invocations.

By implementing deterministic IV generation construction we can get rid of this limit and go back to 2^96

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.