Zmq secure connect implementation #641

AionJayT · 2018-09-14T21:42:36Z

Notice

It is not allowed to submit your PR to the master branch directly, please submit your PR to the master-pre-merge branch.

Description

The implementation allows the zmq server setup the secure connection by the build-in Curve algo.
To enable the secure connection, the user can enable the settings in the config.xml "secure-connect".
Also, the user needs to generate the curve keypair ( by using ./aion.sh -zs). And give the public key to the API client side. The client API need to upgrade to the v0.1.13 or later version.

Fixes Issue # .

Type of change

Insert x into the following checkboxes to confirm (eg. [x]):

Testing

Please describe the tests you used to validate this pull request. Provide any relevant details for test configurations as well as any instructions to reproduce these results.

it has been tested by the wallet team.

Verification

Insert x into the following checkboxes to confirm (eg. [x]):

I have self-reviewed my own code and conformed to the style guidelines of this project.
New and existing tests pass locally with my changes.
I have added tests for my fix or feature.
I have made appropriate changes to the corresponding documentation.
My code generates no new warnings.
Any dependent changes have been made.

arajasek

LGTM :)

AlexandraRoatis · 2018-09-19T13:47:45Z

modAionImpl/src/org/aion/zero/impl/cli/Cli.java

@@ -786,7 +853,7 @@ private static boolean copyRecursively(File src, File target)
        }
        else {
            try {
-                Files.copy(src, target);
+                com.google.common.io.Files.copy(src, target);


is there a reason this was removed from imports?

there are 2 different Files classes using in this class

AlexandraRoatis · 2018-09-19T13:49:12Z

modAionImpl/src/org/aion/zero/impl/cli/Cli.java

+        System.getProperty("user.dir") + File.separator + CfgSsl.SSL_KEYSTORE_DIR);
+
+    private File zmqkeyDir = new File(
+        System.getProperty("user.dir") + File.separator + CfgApiZmq.ZMQ_KEY_DIR);


the zmq_keystore should be added to .gitignore

+1 for Alexandra's suggestion

Also, can we have the value of System.getProperty() be passed into Cli's constructor instead of referring to it directly? The dependence on static methtod calls will make the class hard to test / modularize.

@aion-kelvin created an issue for tracking System.getProperty()
#652

AlexandraRoatis · 2018-09-19T13:52:51Z

modAionImpl/src/org/aion/zero/impl/cli/Cli.java

+                case "-zs":
+                    checkZmqKeystoreDir();
+                    ZMQ.Curve.KeyPair kp = ZMQ.Curve.generateKeyPair();
+                    genKeyFile(kp.publicKey, kp.secretKey);


Based on this implementation a user can generate multiple keys. Does having multiple keys have any advantage? Based on the key loading implementation I see a disadvantage in the fact that the chosen pair is relatively random. It may be preferable to overwrite the old key if the key generation is called a second time.

@AlexandraRoatis

It may be preferable to overwrite the old key if the key generation is called a second time.

Alternatively, refuse to generate another one unless they manually remove the existing one, so they don't accidentally delete their private key and lose it forever. Or just prompt them before overwriting.

AlexandraRoatis · 2018-09-19T13:59:36Z

modApiServer/src/org/aion/api/server/zmq/ProtocolProcessor.java

@@ -144,6 +186,39 @@ public void run() {
        }
    }

+    private void loadCurveKeyPair() {
+        List<File> files = getFiles();


Perhaps it would be useful here to check that the number of files is >= 2.

Moreover, since generating the keys does not require any user input, why not simply generate them for the user when the files are missing? This would make the CLI call to generate the key pair not necessary.

aion-kelvin

Hey looks pretty good ... have a few comments related to refactoring/style

aion-kelvin · 2018-09-24T15:46:13Z

modAionImpl/src/org/aion/zero/impl/cli/Cli.java

+        System.getProperty("user.dir") + File.separator + CfgSsl.SSL_KEYSTORE_DIR);
+
+    private File zmqkeyDir = new File(
+        System.getProperty("user.dir") + File.separator + CfgApiZmq.ZMQ_KEY_DIR);


+1 for Alexandra's suggestion

Also, can we have the value of System.getProperty() be passed into Cli's constructor instead of referring to it directly? The dependence on static methtod calls will make the class hard to test / modularize.

aion-kelvin · 2018-09-24T15:49:19Z

modAionImpl/src/org/aion/zero/impl/cli/Cli.java

+                case "-zs":
+                    checkZmqKeystoreDir();
+                    ZMQ.Curve.KeyPair kp = ZMQ.Curve.generateKeyPair();
+                    genKeyFile(kp.publicKey, kp.secretKey);


@AlexandraRoatis

It may be preferable to overwrite the old key if the key generation is called a second time.

Alternatively, refuse to generate another one unless they manually remove the existing one, so they don't accidentally delete their private key and lose it forever. Or just prompt them before overwriting.

aion-kelvin · 2018-09-24T15:50:16Z

modAionImpl/src/org/aion/zero/impl/cli/Cli.java

+                    ZMQ.Curve.KeyPair kp = ZMQ.Curve.generateKeyPair();
+                    genKeyFile(kp.publicKey, kp.secretKey);
+                    System.out.println("Generate ZmqKeyPairFinished!");
+                    return 1;


I think the convention is that a program returns 0 if it ran successfully, non-zero if it failed. So i think we should return 0 here

aion-kelvin · 2018-09-24T15:51:23Z

modAionImpl/src/org/aion/zero/impl/cli/Cli.java

+        getFile(fileName, secretKey);
+    }
+
+    private void getFile(@Nonnull final String fileName, @Nonnull final String key) throws IOException {


method name is a bit misleading... how about "writeKeyToFile"

aion-kelvin · 2018-09-24T15:53:00Z

modMcf/src/org/aion/mcf/config/CfgApiZmq.java

+                            try {
+                                secureConnectEnabled = Boolean.parseBoolean(Cfg.readValue(sr));
+                            } catch (Exception e) {
+                                //System.out.println("failed to read config node: aion.api.rpc.filters-enabled; using preset: " + this.filtersEnabled);


I think the catch should log a warning

aion-kelvin · 2018-09-24T16:01:37Z

modApiServer/src/org/aion/api/server/zmq/ProtocolProcessor.java

    private static final long zmqHWM = 100_000;
    private static final int SOCKETID_LEN = 5;
    private static final int SOCKET_RECV_TIMEOUT = 3000;

+    static {
+        String storageDir = System.getProperty("local.storage.dir");


Can we do the following to avoid the reliance on the static block:

make it so that storageDir is an argument to the constructor

make CURVEKEY_PATH and PATH non-static

move the logic of setting CURVEKEY_PATH and PATH into constructor

add either another constructor or a static method, that will call the constructor, and pass in `System.getProperty('local.storage.dir') as the value for storageDir

aion-kelvin · 2018-09-24T16:06:49Z

modApiServer/src/org/aion/api/server/zmq/ProtocolProcessor.java

+    private void loadCurveKeyPair() {
+        List<File> files = getFiles();
+        String nextLoad = "";
+        for (File f : files) {


Is there a benefit to this fall-back/fuzzy logic? Why not just have two hard-coded file names it looks for?

I was thinking the user will generate several keypairs in case need to switch the keyset. The public key also needs to be updated to the client side, then the connection will be setup.

modMcf/src/org/aion/mcf/config/CfgApiZmq.java

…d cli return logic fix for secure connect

aion-kelvin

thanks for the changes!

…g settings has been loaded

AionJayT added 9 commits September 6, 2018 17:24

zmq secure connection implement

56a07ca

Merge branch 'native-check' into secure-zmq

9fc903a

implement the curve key loading in the zmq ProtocolProcessor

49c46a7

more detail for the log and the config settings

f994aea

Merge branch 'master' into secure-zmq

56998ef

fix zmq key loading error

91c64b1

fix api secure connect config reset after kernel launch

3d570d6

modify zmq key naming rule for the OS compatibility

17fc73c

Merge branch 'master-pre-merge' into secure-zmq

8067f6f

AionJayT added the feature label Sep 14, 2018

AionJayT added this to the 0.3.2 milestone Sep 14, 2018

AionJayT requested review from arajasek, AlexandraRoatis and aion-kelvin September 14, 2018 21:42

arajasek approved these changes Sep 18, 2018

View reviewed changes

AlexandraRoatis suggested changes Sep 19, 2018

View reviewed changes

aion-kelvin suggested changes Sep 24, 2018

View reviewed changes

AionJayT and others added 6 commits September 25, 2018 15:08

add zmq_keystore into gitignore

49c0ea7

refactoring the getFiles method to a class

3f84a20

refactoring zmq secure connection for logging

1c75109

refactoring CfgApiZmq logging, ProtocolProcessor construct process an…

33c68bd

…d cli return logic fix for secure connect

fix merge error

0f14ba4

update api repo ref

04672de

aion-kelvin approved these changes Oct 2, 2018

View reviewed changes

aionjay added 2 commits October 2, 2018 15:44

move zmq key pair generate ouside of Cli. Will be checked after confi…

415caea

…g settings has been loaded

undo format Cli and remove comment in Aion class

27caad4

AlexandraRoatis approved these changes Oct 2, 2018

View reviewed changes

aionjay added 3 commits October 2, 2018 16:33

fix pack issue

a51c944

Merge branch 'master-pre-merge' into secure-zmq

117a502

fix module-info settings for modAionImpl

699080a

update default config settings for the Java API secure connect

2208f82

AionJayT merged commit 4aed4ca into master-pre-merge Oct 3, 2018

AionJayT deleted the secure-zmq branch October 4, 2018 17:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Zmq secure connect implementation #641

Zmq secure connect implementation #641

AionJayT commented Sep 14, 2018

arajasek left a comment

AlexandraRoatis Sep 19, 2018

AionJayT Sep 19, 2018

AlexandraRoatis Sep 19, 2018

aion-kelvin Sep 24, 2018

AionJayT Sep 25, 2018 •

edited

Loading

AlexandraRoatis Sep 19, 2018

aion-kelvin Sep 24, 2018

AlexandraRoatis Sep 19, 2018

AlexandraRoatis Sep 19, 2018 •

edited

Loading

aion-kelvin left a comment

aion-kelvin Sep 24, 2018

aion-kelvin Sep 24, 2018

aion-kelvin Sep 24, 2018

aion-kelvin Sep 24, 2018

aion-kelvin Sep 24, 2018

aion-kelvin Sep 24, 2018

aion-kelvin Sep 24, 2018

AionJayT Sep 25, 2018

aion-kelvin left a comment

Zmq secure connect implementation #641

Zmq secure connect implementation #641

Conversation

AionJayT commented Sep 14, 2018

Notice

Description

Type of change

Testing

Verification

arajasek left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AionJayT Sep 25, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AlexandraRoatis Sep 19, 2018 • edited Loading

Choose a reason for hiding this comment

aion-kelvin left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aion-kelvin left a comment

Choose a reason for hiding this comment

AionJayT Sep 25, 2018 •

edited

Loading

AlexandraRoatis Sep 19, 2018 •

edited

Loading