Skip to content

Commit

Permalink
repackages java class deps with jarjar links
Browse files Browse the repository at this point in the history
- applies jarjar links during source-deps phase
  - after munging clj sources jar all class files into one jar
  - apply jarjar links on this jar file
  - remove directories from target/srcdeps containing class files
  - unzip jarjar links output jar file into target/srcdeps
  - repackage prefix is generated from projectname and project version
- util fn to figure out where to find java class files.
- some jarjar helpers inlined so they are accessible properly
- main processor is stripped so it only does packag remapping
- cleans up logging msgs
- gets rid of eval-in
- jumps to 0.4.0
  - updates readme
- prefixing in transient deps
- takes care of class references in namespace body
- cleans up
  - args handling
  - readme
  - uses the latest mranderson
  • Loading branch information
benedekfazekas committed Mar 31, 2015
1 parent b4ba165 commit 6332fef
Show file tree
Hide file tree
Showing 8 changed files with 554 additions and 44 deletions.
40 changes: 20 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,17 @@ Dependencies as source: used as if part of the project itself.

Somewhat node.js & npm style dependency handling as a leiningen plugin.

**Fancy words: 'npm style dependency handling' but what is this project is really about?**

It basically makes inlining a less burdensome task. It automatically retrieves and prefixes your dependencies (both clojure source and java class files) and munges your clojure files -- mainly the namespace declaration but not only -- accordingly.

## Prerequisites

**IMPORTANT** You need to install leiningen version **2.5.0** at least for this plugin if you want to use the built in profile (see below for explanation). Basic functionality (eg `lein source-deps`) still works though.

## Usage

Put `[thomasa/mranderson "0.3.0"]` into the `:plugins` vector of your project.clj.
Put `[thomasa/mranderson "0.4.0"]` into the `:plugins` vector of your project.clj.

Additionally you also need to mark some of the dependencies in your dependencies vector in the project's `project.clj` with `^:source-dep` meta tag. For example:

Expand All @@ -25,50 +29,46 @@ Now you are ready to run:

$ lein source-deps

**What happens here?**
this retrieves dependencies and creates a deeply nested directory structure for them in `target/srcdeps` directory. It also munges all clojure source files accordingly. More over it uses [Jar Jar Links](https://code.google.com/p/jarjar/) to repackage your java class files dependencies if any.

The plugin basically retrieves the dependencies you marked (and their transitive dependencies) and builds a nested tree of directories in `target/srcdeps` directory with the retrieved files. It also copies the project's sources under this directory and modifies both the project source files and the dependency source files so their namespace names are all reflecting the newly created nested directory structure.
If you don't want mranderson to repackage your java dependencies you can opt out by passing `skip-javaclass-repackage true` as a parameter to `source-deps` task.

**So far so good, but what's then?**
After that you can run your tests or your repl with:

Yes, this is only half way. Now you can of course still work with your original source tree but the plugin also provides you a built in profile which enables you to work with the munged sourcetree including your dependencies. For example you can (and should) run your tests with the modified source tree:
$ lein with-profile +plugin.mranderson/config repl

$ lein with-profile +plugin.mranderson/config test

Please note the plus sign before the mranderson profile. This does not stop here of course you can start up your repl with the modified source tree too of course.
note the plus sign before the leiningen profile.

$ lein with-profile +plugin.mranderson/config repl
If you want to use mranderson while developing locally with the repl the source has to be modified in the target/srcdeps directory.

When you want to release locally:

**Ok I can play with the modified source but how do I release?**
$ lein with-profile plugin.mranderson/config install

The usual way only use the above mentioned built in profile when you run `jar`, `install` and friends.
to clojars:

**What happens when I upgrade one of the depencies?**
$ lein with-profile +plugin.mranderson/config deploy clojars

Easy: run
If you want to change, update your dependencies just edit your `project.clj` file the usual way and run

$ lein clean

change your dependencies and then again
and then again

$ lein source-deps

and you are good to go.

**I deployed to clojars and my deployed jar is not mrandersoned. What happened?**

As `deploy clojars` rebuilds the jar and the pom you have to use the profile here too:

$ lein with-profile plugin.mranderson/config deploy clojars

otherwise an old style jar will be built and uploaded.

**note** you should not mark clojure itself as a source dependency: there is a limit for everything.

## Under the hood

There is not much magic there but simple modifing the source files as strings [tools.namespace](https://github.com/clojure/tools.namespace) style; see specially `clojure.tools.namespace.move` namespace. Also additionally some more source file munging is done for prefixes, some deftypes and the like.

It also uses [Jar Jar Links](https://code.google.com/p/jarjar/) to repackage your java class files dependencies if any.

A bit of additional magic happens when you use the built in profile: it actively switches on the leiningen middleware also built into the plugin. The middleware AOT compiles some clojure sources and removes dependencies marked with `^:source-deps` from the dependency list. So they won't show up in the generated pom file and so on.

## Rationale
Expand Down
85 changes: 85 additions & 0 deletions java-src/mranderson/util/JjMainProcessor.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
/**
* Copied from Jar Jar Links 1.4
*
* Original licence
*
* Copyright 2007 Google Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package mranderson.util;

import com.tonicsystems.jarjar.*;
import com.tonicsystems.jarjar.ext_util.*;
import java.io.File;
import java.io.IOException;
import java.util.*;

public class JjMainProcessor implements JarProcessor
{
private final boolean verbose;
private final JarProcessorChain chain;
private final Map<String, String> renames = new HashMap<String, String>();

public JjMainProcessor(List<PatternElement> patterns, boolean verbose, boolean skipManifest) {
this.verbose = verbose;
List<Rule> ruleList = new ArrayList<Rule>();
for (PatternElement pattern : patterns) {
if (pattern instanceof Rule) {
ruleList.add((Rule) pattern);
}
}

JjPackageRemapper pr = new JjPackageRemapper(ruleList, verbose);

List<JarProcessor> processors = new ArrayList<JarProcessor>();
processors.add(new JarTransformerChain(new RemappingClassTransformer[]{ new RemappingClassTransformer(pr) }));
chain = new JarProcessorChain(processors.toArray(new JarProcessor[processors.size()]));
}

public void strip(File file) throws IOException {
return;
}

/**
* Returns the <code>.class</code> files to delete. As well the root-parameter as the rename ones
* are taken in consideration, so that the concerned files are not listed in the result.
*
* @return the paths of the files in the jar-archive, including the <code>.class</code> suffix
*/
private Set<String> getExcludes() {
return new HashSet<String>();
}

/**
*
* @param struct
* @return <code>true</code> if the entry is to include in the output jar
* @throws IOException
*/
public boolean process(EntryStruct struct) throws IOException {
String name = struct.name;
boolean keepIt = chain.process(struct);
if (keepIt) {
if (!name.equals(struct.name)) {
if (verbose)
System.err.println("Renamed " + name + " -> " + struct.name);
}
} else {
if (verbose)
System.err.println("Removed " + name);
}
return keepIt;
}
}
145 changes: 145 additions & 0 deletions java-src/mranderson/util/JjPackageRemapper.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,145 @@
/**
* Copied from Jar Jar Links 1.4
*
* Original licence
*
* Copyright 2007 Google Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package mranderson.util;

import com.tonicsystems.jarjar.*;
import com.tonicsystems.jarjar.asm.*;
import com.tonicsystems.jarjar.asm.commons.*;
import java.util.*;
import java.util.regex.Pattern;

public class JjPackageRemapper extends Remapper
{
private static final String RESOURCE_SUFFIX = "RESOURCE";

private static final Pattern ARRAY_FOR_NAME_PATTERN
= Pattern.compile("\\[L[\\p{javaJavaIdentifierPart}\\.]+?;");

private final List<JjWildcard> wildcards;
private final Map<String, String> typeCache = new HashMap<String, String>();
private final Map<String, String> pathCache = new HashMap<String, String>();
private final Map<Object, String> valueCache = new HashMap<Object, String>();
private final boolean verbose;

private static List<JjWildcard> createWildcards(List<? extends PatternElement> patterns) {
List<JjWildcard> wildcards = new ArrayList<JjWildcard>();
for (PatternElement pattern : patterns) {
String result = (pattern instanceof Rule) ? ((Rule)pattern).getResult() : "";
String expr = pattern.getPattern();
if (expr.indexOf('/') >= 0)
throw new IllegalArgumentException("Patterns cannot contain slashes");
wildcards.add(new JjWildcard(expr.replace('.', '/'), result));
}
return wildcards;
}

public JjPackageRemapper(List<Rule> ruleList, boolean verbose) {
this.verbose = verbose;
wildcards = createWildcards(ruleList);
}

// also used by KeepProcessor
static boolean isArrayForName(String value) {
return ARRAY_FOR_NAME_PATTERN.matcher(value).matches();
}

public String map(String key) {
String s = typeCache.get(key);
if (s == null) {
s = replaceHelper(key);
if (key.equals(s))
s = null;
typeCache.put(key, s);
}
return s;
}

public String mapPath(String path) {
String s = pathCache.get(path);
if (s == null) {
s = path;
int slash = s.lastIndexOf('/');
String end;
if (slash < 0) {
end = s;
s = RESOURCE_SUFFIX;
} else {
end = s.substring(slash + 1);
s = s.substring(0, slash + 1) + RESOURCE_SUFFIX;
}
boolean absolute = s.startsWith("/");
if (absolute) s = s.substring(1);

s = replaceHelper(s);

if (absolute) s = "/" + s;
if (s.indexOf(RESOURCE_SUFFIX) < 0)
return path;
s = s.substring(0, s.length() - RESOURCE_SUFFIX.length()) + end;
pathCache.put(path, s);
}
return s;
}

public Object mapValue(Object value) {
if (value instanceof String) {
String s = valueCache.get(value);
if (s == null) {
s = (String)value;
if (isArrayForName(s)) {
String desc1 = s.replace('.', '/');
String desc2 = mapDesc(desc1);
if (!desc2.equals(desc1))
return desc2.replace('/', '.');
} else {
s = mapPath(s);
if (s.equals(value)) {
boolean hasDot = s.indexOf('.') >= 0;
boolean hasSlash = s.indexOf('/') >= 0;
if (!(hasDot && hasSlash)) {
if (hasDot) {
s = replaceHelper(s.replace('.', '/')).replace('/', '.');
} else {
s = replaceHelper(s);
}
}
}
}
valueCache.put(value, s);
}
// TODO: add back class name to verbose message
if (verbose && !s.equals(value))
System.err.println("Changed \"" + value + "\" -> \"" + s + "\"");
return s;
} else {
return super.mapValue(value);
}
}

private String replaceHelper(String value) {
for (JjWildcard wildcard : wildcards) {
String test = wildcard.replace(value);
if (test != null)
return test;
}
return value;
}
}
Loading

0 comments on commit 6332fef

Please sign in to comment.