Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat!: switch to arsclib #165

Closed
wants to merge 8 commits into from
Closed

feat!: switch to arsclib #165

wants to merge 8 commits into from

Conversation

Axelen123
Copy link
Member

@Axelen123 Axelen123 commented Apr 26, 2023

Fixes #73

Motivation

aapt2 is slow, and has several skill issues such as the inability to run properly on most armv7 devices and sometimes fails to decode properly as seen here.

ARSCLib is faster, and does not experience these issues.

@Axelen123 Axelen123 marked this pull request as ready for review April 27, 2023 16:15
@oSumAtrIX oSumAtrIX changed the base branch from dev to feat/split April 27, 2023 19:18
Copy link
Member

@oSumAtrIX oSumAtrIX left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Please finish the missing documentation.
  • Some classes and implementations feel like they don't belong in ReVanced Patcher, mainly because they only serve the purpose of handling resources. The package app.revanced.patcher.arsc, for example, mainly hosts code to read and write resources. In contrast, only a fraction is necessary for ReVanced Patcher being the class ReVancedApkArchive at max.

build.gradle.kts Outdated Show resolved Hide resolved
build.gradle.kts Outdated Show resolved Hide resolved
build.gradle.kts Outdated Show resolved Hide resolved
Comment on lines 36 to 38
classFile.realName
.let { it.substring(0, it.length - (CLASS_FILE_EXTENSION_LENGTH + 1)) } // remove file extension
.replace('/', '.') // file path to package name
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, this is a hack. Can a specification be relied on to filter a JAR archive for class files?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know of any specifications for this. I have changed it so it is closer to the way it was before.

src/main/kotlin/app/revanced/patcher/apk/ResourceFile.kt Outdated Show resolved Hide resolved
Comment on lines 38 to 41
fun readText() = String(contents)
fun writeText(string: String) {
contents = string.toByteArray()
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do these methods exist?

Copy link
Member Author

@Axelen123 Axelen123 Apr 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh right these make more sense as extension functions in the patches repo. I will move them.

Patches used the read/writeText extension functions of java.io.File to perform string manipulation on XML files. That is why they exist

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A resource file is not yet an XML file. Justifying the existence of these methods by saying resource files are XML files wouldn't be really correct. Classes such as String() should not be visible at patch level as they are low level classes for resources. For that reason moving the function readText to patches should be reconsidered.

Copy link
Member Author

@Axelen123 Axelen123 Apr 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There can be other text files such as json in an apk too, and those are also text. The current implementation just transparently decodes/encodes android binary xml to human readable xml, just like aapt does. Do you think there is there a problem with that?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am highlighting your intention to move the methods to the patches, therefore exposing the String() class publicly to the patches, whereas it should not be visible at the patch level as they are low-level classes for resources. This is why moving the function readText to patches should be reconsidered

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not entirely sure what you mean. readText is just there to read the entire resource file as a normal kotlin string. String is not a custom class or anything.

src/main/kotlin/app/revanced/patcher/Context.kt Outdated Show resolved Hide resolved
src/main/kotlin/app/revanced/patcher/apk/Apk.kt Outdated Show resolved Hide resolved
src/main/kotlin/app/revanced/patcher/apk/Apk.kt Outdated Show resolved Hide resolved
@oSumAtrIX oSumAtrIX changed the base branch from feat/split to dev April 27, 2023 20:39
}
}

return if (resPath.endsWith(".xml")) ResourceFileImpl.XML(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is loose. Resource files could also end without an extension.

Copy link
Member Author

@Axelen123 Axelen123 Apr 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It now reads existing files in order to see if they are AXML or not and only uses file extension when creating new files. Is this sufficient?

@Sculas
Copy link
Contributor

Sculas commented Apr 29, 2023

Shouldn't this branch merge into the feat/split branch? Either that or close #113 since this PR would supersede that one.

@Axelen123
Copy link
Member Author

Axelen123 commented Apr 30, 2023

Shouldn't this branch merge into the feat/split branch? Either that or close #113 since this PR would supersede that one.

Split branch is out of date and this PR supersedes it. #113 should be closed if this PR gets merged

build.gradle.kts Outdated Show resolved Hide resolved
@Axelen123 Axelen123 requested a review from oSumAtrIX May 3, 2023 08:41
src/main/kotlin/app/revanced/patcher/Context.kt Outdated Show resolved Hide resolved
src/main/kotlin/app/revanced/patcher/Context.kt Outdated Show resolved Hide resolved
src/main/kotlin/app/revanced/patcher/Context.kt Outdated Show resolved Hide resolved
src/main/kotlin/app/revanced/patcher/Patcher.kt Outdated Show resolved Hide resolved
src/main/kotlin/app/revanced/patcher/Patcher.kt Outdated Show resolved Hide resolved
src/main/kotlin/app/revanced/patcher/apk/Apk.kt Outdated Show resolved Hide resolved
src/main/kotlin/app/revanced/patcher/apk/Apk.kt Outdated Show resolved Hide resolved
src/main/kotlin/app/revanced/patcher/apk/Apk.kt Outdated Show resolved Hide resolved
src/main/kotlin/app/revanced/patcher/apk/Apk.kt Outdated Show resolved Hide resolved
src/main/kotlin/app/revanced/patcher/apk/Apk.kt Outdated Show resolved Hide resolved
Copy link
Contributor

@Sculas Sculas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's all from me for now. I haven't looked through everything thoroughly, but I'd say at least good enough for a proper review.

src/main/kotlin/app/revanced/patcher/Context.kt Outdated Show resolved Hide resolved
src/main/kotlin/app/revanced/patcher/Context.kt Outdated Show resolved Hide resolved
src/main/kotlin/app/revanced/patcher/Context.kt Outdated Show resolved Hide resolved
src/main/kotlin/app/revanced/patcher/PatcherContext.kt Outdated Show resolved Hide resolved
Comment on lines 11 to 16
* The result of a patch.
*
* @param apk The patched [Apk] file.
*/
sealed class Patch(val apk: Apk) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that doesn't make sense. This is not the result of a single Patch, this is the finalized APK "bundle" the patcher came up with. Please adjust this naming accordingly, such as PatchedApk.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That naming suggests that PatchedApk is Apk which isn't correct, please suggest another name

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Sculas do you have a better name?

}
}

inner class Resources(val tableBlock: TableBlock?) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about ResourceContainer?

Suggested change
inner class Resources(val tableBlock: TableBlock?) {
inner class ResourceContainer(val tableBlock: TableBlock?) {

Copy link
Contributor

@Sculas Sculas May 4, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my earlier comment about PatchException: #165 (comment)

src/main/kotlin/app/revanced/patcher/util/dom/DomUtil.kt Outdated Show resolved Hide resolved
@oSumAtrIX oSumAtrIX deleted the branch ReVanced:dev May 5, 2023 23:05
@oSumAtrIX oSumAtrIX closed this May 5, 2023
@Sculas
Copy link
Contributor

Sculas commented May 5, 2023

@oSumAtrIX This one too.

@oSumAtrIX oSumAtrIX reopened this May 6, 2023
@Sculas
Copy link
Contributor

Sculas commented May 6, 2023

PR should be renamed to a breaking change, btw.

@Axelen123 Axelen123 changed the title feat: switch to arsclib feat!: switch to arsclib May 6, 2023
@Axelen123 Axelen123 changed the title feat!: switch to arsclib feat: switch to arsclib May 6, 2023
@Axelen123 Axelen123 changed the title feat: switch to arsclib feat!: switch to arsclib May 6, 2023
@Axelen123
Copy link
Member Author

I have added documentation and moved some code around.

Almost all the other reviews are marked as resolved, which is why I am requesting another one.

build.gradle.kts Outdated Show resolved Hide resolved
{ file.close() })

/**
* Closes the editor. Write backs and decreases the lock count.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* Closes the editor. Write backs and decreases the lock count.
* Closes the editor, writes back the file, and decreases the lock count.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is lock count still up to date here.

src/main/kotlin/app/revanced/patcher/Context.kt Outdated Show resolved Hide resolved
/**
* Wrapper for a file that can be edited as a dom document.
*
* This constructor does not check for locks to the file when writing.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see any related code to locking

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like I forgot to remove the documentation related to locking after I remove the lock itself.
I removed those docs and simplified the save code because we don't have to do any OutputStream workarounds unlike with real files.

src/main/kotlin/app/revanced/patcher/Context.kt Outdated Show resolved Hide resolved
src/main/kotlin/app/revanced/patcher/apk/ApkBundle.kt Outdated Show resolved Hide resolved
src/main/kotlin/app/revanced/patcher/apk/ApkBundle.kt Outdated Show resolved Hide resolved
src/main/kotlin/app/revanced/patcher/apk/ApkBundle.kt Outdated Show resolved Hide resolved
SplitApkResult(it, file, exception)
}

inner class GlobalResources {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this class part of an ApkBundle and what is it used for? Please document it.

}

/**
* The global resource container.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please clarify what a global resource container for ApkBundle means.

@Axelen123
Copy link
Member Author

@oSumAtrIX the commits you pushed look good to me other than the PatchBundle bug which I fixed.
I have also made the PatchBundle classes accept a File instead of a string locally and I plan on pushing that soon.

BREAKING CHANGE: This removes the previously available `Path` option
Comment on lines 20 to 18
publishing {
repositories {
if (System.getenv("GITHUB_ACTOR") != null)
maven {
name = "GitHubPackages"
url = uri("https://maven.pkg.github.com/revanced/revanced-patcher")
credentials {
username = System.getenv("GITHUB_ACTOR")
password = System.getenv("GITHUB_TOKEN")
}
}
else
mavenLocal()
}
publications {
register<MavenPublication>("gpr") {
from(components["java"])
}
}
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does the arsclib module depend on patcher?

@oSumAtrIX oSumAtrIX force-pushed the insan branch 2 times, most recently from 1fce0c8 to 1b543b0 Compare July 15, 2023 18:10
@oSumAtrIX
Copy link
Member

If possible, introduce some tests to the arsclib-utils module.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file should be in the root folder

* @param handle The [Handle] associated with this file.
* @param archive The [Archive] that the file resides in.
*/
class ResourceFile private constructor(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file should be removed in favor of the new class ArchiveResource. You use dependency inversion to inject the archive this resource belongs to and write back to it. This can be implemented in ArchiveResource too. Is it necessary to invert the dependency to Handle? Also, ArchiveResource is abstracted in a way that does not need to read all bytes to a byte array. Before my changes, you were reading all bytes from a parsed XmlDocument file, then later on parsing it again in DomFIleEditor as an w3c XML file. w3c XML files should be removed, and XmlDocument from ARSCLib should be used instead. This way, we stop depending on w3c XML as well as don't have to do all the unnecessary writing and parsing

@@ -0,0 +1,7 @@
package app.revanced.arsc.logging
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe relying on a logging facade is better than having these interfaces in patcher and arsclib-utils.

Comment on lines +1 to +26
pluginManagement {
repositories {
mavenCentral()
maven {
url = uri("https://maven.pkg.github.com/revanced/multidexlib2")
credentials {
username = providers.gradleProperty("gpr.user").orNull ?: System.getenv("GITHUB_ACTOR")
password = providers.gradleProperty("gpr.key").orNull ?: System.getenv("GITHUB_TOKEN")
}
}
}
}

dependencyResolutionManagement {
repositoriesMode.set(RepositoriesMode.FAIL_ON_PROJECT_REPOS)
repositories {
mavenCentral()
maven {
url = uri("https://maven.pkg.github.com/revanced/multidexlib2")
credentials {
username = providers.gradleProperty("gpr.user").orNull ?: System.getenv("GITHUB_ACTOR")
password = providers.gradleProperty("gpr.key").orNull ?: System.getenv("GITHUB_TOKEN")
}
}
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the reason for this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not entirely sure. I just used compose manager as a reference when setting up the gradle monorepo because android projects are always setup like that.

/**
* A set of all architectures supported by android.
*/
val architectures = setOf("armeabi_v7a", "arm64_v8a", "x86", "x86_64")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, the architectures are hard coded. What do you think of propagating the selection of the correct Apk types? This means the patcher would not automatically know the library APK architecture and instead require the library user to input the correct one. The benefit of that would be that the architecture wouldn't be hardcoded anymore.

Copy link
Member Author

@Axelen123 Axelen123 Jul 17, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That just contains a set containing all architectures supported by android (not the current/target device). I don't think that list is likely to change any time soon so hardcoding probably isn't really an issue. It is used for determining if the split configuration is valid or not, so we cannot rely on the list of architectures supported by the current device (SUPPORTED_ABIS on android).

Adding the ability for frontends to filter which apks are used might make sense though.

Comment on lines +82 to +88
val module = manifestElement.searchAttributeByName("configForSplit")
?.let { Module.DynamicFeature(it.valueAsString) } ?: Module.Main

// Examples:
// config.xhdpi
// df_my_feature.config.en
val config = this.split.split(".").last()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are those conventions written in stone or can a language APKModule not end with a two-letter country code?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Android uses ISO 639-1:2002 language codes and those are always two letters long.

Language variants (en_US, en_GB) are just stored in one apk (en).

@@ -0,0 +1,42 @@
.gradle
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file should be in the root of the repository

@yoshiask

This comment was marked as spam.

@Ushie
Copy link
Member

Ushie commented Aug 11, 2023

Iirc it's considered 20% done, there's further refactoring and things necessary

@Axelen123
Copy link
Member Author

I am going to close this for two reasons:

  • Maintainer wants a rewrite
  • Keeping the branches up-to-date (especially patches) is a burden that I no longer have the time or motivation to handle

Anyone who wants to give this a shot can use my branch as a reference.

@Axelen123 Axelen123 closed this Sep 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants