-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Issue #19: Module Component #33
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe it is better to do that in a separate PR.
Separate commit is fine, can be minor.
If you have any concerns/questions, feel free to ask.
@@ -0,0 +1,2340 @@ | |||
[ | |||
{ | |||
"name": "Checker", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When I said hardcoded list, I meant like PackageObjectFactory
did at https://github.com/checkstyle/checkstyle/blob/b024af7dbc3f53863e17147692100381f4165073/src/main/java/com/puppycrawl/tools/checkstyle/PackageObjectFactory.java#L338 , as we are not sure yet if it will be a file with XML/JSON or something else.
It can stay for now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
config/import-control.xml
Outdated
@@ -7,7 +7,18 @@ | |||
<allow pkg="java.io"/> | |||
<allow pkg="java.util"/> | |||
<allow pkg="com.github.checkstyle.regression.data"/> | |||
<allow pkg="com.github.checkstyle.regression.util"/> | |||
<allow pkg="org.immutables"/> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add space after this item, and between each subpackage
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done.
config/import-control.xml
Outdated
@@ -7,7 +7,18 @@ | |||
<allow pkg="java.io"/> | |||
<allow pkg="java.util"/> | |||
<allow pkg="com.github.checkstyle.regression.data"/> | |||
<allow pkg="com.github.checkstyle.regression.util"/> | |||
<allow pkg="org.immutables"/> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
org.immutables
. All this just to have it create the class for us?
I am not a fan of auto-generated classes as you can't see the code for it. We also lost code coverage for it, so we can't tell if a field is ever used or if a method definition covers all branches.
Why do we need it for simple POJOs when we don't do this for the other POJOs in git
component?
I will get back to you on this item.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Luolc I asked romani his opinion on this, and he isn't a fan of "magic" builders either.
Since this isn't the main repository, we could keep this here if we really wanted to.
My only concern right now is code coverage as ModuleInfo
isn't showing any or requiring any, even for hasCheckerParent
which is a branch because of the equality test.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe there would be many fields in ModuleInfo
in the future.
If we use constructor to pass these parameters, it would be ugly. 《Effective Java》 said it is better to use builder style if constructor has more than three parameters.
Another annoying thing is the equals
method. We would have a lot of equals
asserting in UTs. It is hard to appease codecov if we write this manually. Supposed there are 5 fields then the equals
method would have 32 branches(2^5). That's horrible. If we write a equals
method in UTs, yeah that could be a workaround, be a little bit strange. And it is also bad that when we add or change some fields, then we have to update the utility equals
in UTs.
So I think it is not bad if an annotation processing tool could generate these for us. We could then focus on more important things but not writing builders, equals etc.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Supposed there are 5 fields then the equals method would have 32 branches(2^5). That's horrible.
I assume we got around this in main project using equalsverifier
dependency.
So I think it is not bad if an annotation processing tool could generate these for us
Let's keep org.immutables
than, but how about we require no custom methods in the interface and require them to be in utility
classes so we can keep code coverage on them? I dislike the idea of having code we can't be sure we are covering.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Keeping them as abstract classes seems to keep them requiring coverage, so lets stick with this.
This is done.
* Contains the logic of regression config generation. | ||
* @author LuoLiangchen | ||
*/ | ||
public final class ConfigGenerator { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IMO, this is a bad name for a class in the module
component, especially since we will have another component called configuration
. This will easily confuse people and they question why we don't have ConfigGenerator
in configuration
component.
We need a more suitable name for this and generateConfigNodes
.
* @param changes the changes source | ||
* @return the regression config nodes generated from the given changes | ||
*/ | ||
public static List<ConfigNode> generateConfigNodes(List<GitChange> changes) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not sure if this component should return ConfigNode
.
When we transverse through the changes, modules and tests will be in separate files, but we have to identify them together and build multiple modules from the data in the 2.
Will ConfigNode
cause a lot of extra searching if we leave it in this format? How will you maintain list of properties and values to cycle through?
Would a map of module names and a custom class of their retrieved data be better ? Would create the actual configuration hierarchy in configuration
component.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it depends on the duty scope of configuration
. I was thinking that configuration
would only get direct data describing the XML.(module name, property name, value etc.) If we are going to pass more data to configuration
, then how detailed it would be?
I guess I misunderstood some duties of these two component.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The only duty of configuration
is build the configuration file. There are no specifics on the type of data it should retrieve.
Someone has to build the configuration, and someone has to generate the list of different modules to build based on properties and such. It's not a concern who does which, but do we need an intermediate data type before the direct data describing the XML
?
I was thinking that configuration would only get direct data describing the XML.
My reasoning against this is "what if we have to build a new component between module and configuration". Can this new component deal with the design structure of the final configuration? Would it just want to see a single module as a whole?
We can hold this off for now, but I am just worried about if this structure will be good for #5 as we need to create multiple module configurations for each set of properties.
Which is why I think the answer to How will you maintain list of properties and values to cycle through?
is an important question.
*/ | ||
public final class ModuleUtils { | ||
/** The compiled regex pattern of the path of Java main source files. */ | ||
private static final Pattern MAIN_SOURCE_PATTERN = Pattern.compile("src/main/java/(.+)\\.java"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will windows require \
? If so, we may need [\\/]
like our suppressions.
I will look into this and get back.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ConfigGeneratorTest
fails when I change all file paths from /
to \\
.
I used another utility I wrote with jgit to see how it returns files and folders. DiffEntry
's paths are returned with /
in Windows.
So it seems this won't be an issue and can be ignored for now.
} | ||
} | ||
catch (IOException ex) { | ||
throw new IllegalStateException("Failed when loaing hardcode information", ex); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hardcode information
=> hardcoded checkstyle module information
Best to be as specific as possible.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
static { | ||
final GsonBuilder gsonBuilder = new GsonBuilder(); | ||
for (TypeAdapterFactory factory : ServiceLoader.load(TypeAdapterFactory.class)) { | ||
gsonBuilder.registerTypeAdapterFactory(factory); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What are these 2 lines doing?
Can and should this be embedded with the other file stuff?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is for registering the GsonTypeAdapter generated by immutables. It is sort of initialization of Gson.
FYI: https://immutables.github.io/json.html#type-adapter-registration
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As discussed below, lets keep all gson
stuff together in one class.
} | ||
|
||
@Test | ||
public void testGenerateConfigNodes() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Split this into 2 tests. Invalid and valid changes. (valid = recognize as checkstyle module)
Add a new invalid location inside JAVA_MAIN_SOURCE_PREFIX
and inside src\main\java
but both not recognized as a module. This is an example of a checkstyle utility class or a brand new module that we didn't recognize.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I split the current UT into two. Not very clear about "new invalid" mean. And what does "inside" mean?
Added UT:
@Test
public void testGenerateConfigNodesForInvalidChanges() {
final List<GitChange> changes = new LinkedList<>();
changes.add(new GitChange(
JAVA_MAIN_SOURCE_PREFIX + "PackageObjectFactory.java"));
changes.add(new GitChange("foo/A.java"));
final List<ConfigNode> configNodes = ConfigGenerator.generateConfigNodes(changes);
assertEquals(0, configNodes.size());
}
a new invalid location inside
JAVA_MAIN_SOURCE_PREFIX
I guess changes.add(new GitChange(JAVA_MAIN_SOURCE_PREFIX + "PackageObjectFactory.java"));
is this situation.
inside
src\main\java
Do you mean smth like src/main/java/Foo.java
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"new invalid" mean
I just meant a new file/folder that you hadn't used before.
"inside" mean
A file inside that folder location.
I guess ...JAVA_MAIN_SOURCE_PREFIX + "PackageObjectFactory.java"
Yes.
Do you mean smth like src/main/java/Foo.java?
Yes.
final InputStream is = spy(ModuleUtils.class.getClassLoader() | ||
.getResourceAsStream("checkstyle_modules.json")); | ||
doThrow(IOException.class).when(is).close(); | ||
try { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
empty line before try.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
import com.github.checkstyle.regression.data.ImmutableConfigNode; | ||
|
||
public class ConfigGeneratorTest { | ||
private static final String BASE_PACKAGE = "com.puppycrawl.tools.checkstyle"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Field is never used.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removed.
5530428
to
607235b
Compare
config/import-control.xml
Outdated
</subpackage> | ||
|
||
<subpackage name="util"> | ||
<allow pkg="com.google.gson"/> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure what happened to my original post on this as I can't find it anymore.
I am fine with util
package, but it seems to me we are breaking component design by using component specific com.google.gson
outside of module
component, which is the only component that will use it right?
Should this be a util
inside module?
It also seems like ModuleUtils
is trying to do too much. It does json reading and other various things.
I think we should have one utility for JSON, including code in GsonProvider
, and have another class that is what utility methods module needs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Luolc Please see this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Lets move all gson code to new component in #37 .
Either this component will now receive the collection of ModuleExtractInfos, or a utility class will and keep it as global information.
@Luolc ping |
6c42f41
to
9c0b260
Compare
|
I don't think this class can be fully immutable. How are you going to combine different changes in separate Here is what I recommend:
When we go to next component, it will take Does this make sense? |
I update the definition of
What if in a PR there is only one test file changed? In this case we don't need regression. We have many issues that only need to change test scope without touching production code. IMO, we should go through the production code changes first, adding some modules to the map(direct change now, and gain related modules from utility changes in the future). Then, go through the test changes. If the map contains the corresponding module already, then process the test change, otherwise just ignore it. |
Agreed. |
Updated according to the suggestions. Please have a check. |
@Luolc Everything is looking good. Just to summarize: Lets move all gson code to new component in #37 . You can make a new PR now with just this code if you wish but I still prefer it in one file. Lets remove Remove commented code. If it is needed for future items, keep them as a copy somewhere. Coverage can be less than 100% for now to get this merged faster. Split dependency re-ordering into minor commit. |
Don't have questions on others |
Answered in issue #37 . |
Split to #38 Then changes about adding objects to
In #40 , also, blocked by #38 and #39 Other changes are also done and pushed, but need these PRs above are merged and rebase them. |
Please rebase to remove conflicts. Only changes for main component should remain. |
fe701c4
to
a26ba49
Compare
Rebased and pushed. I am adding the assert messages in test now. Edit: I uncommented the codes about test and utility classes in |
* Checks whether the corresponding file of a change may be considered as | ||
* a Java main source file. | ||
* @param change change to check | ||
* @return true if the corresponding file of a change may be considered as |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
duplicating main description in return tag is bad javadoc design. You are not giving reader anything new to read.
Can you try to fix these and others by making each unique and different information and not just a rewording?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I add some info in each main description. Refers to the similar utility methods in ModuleReflectionUtils
in checkstyle project.
*/ | ||
public static boolean isCheckstyleModuleTest(GitChange change) { | ||
final boolean returnValue; | ||
if (JAVA_SOURCE_PARTTEN.matcher(change.getPath()).find()) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is true
for test and main. It should only be accepting test. Make a isJavaTestSource
and use that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
* Sets the map of full qualified name to module extract info with the given map. | ||
* @param map the given map | ||
*/ | ||
public static void setNameToModuleExtratInfo(Map<String, ModuleExtractInfo> map) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Extrat
=> Extract
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
* @param change the change to convert | ||
* @return the full name of corresponding checkstyle module | ||
*/ | ||
public static String convertModuleTestChangeToModuleFullName(GitChange change) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is no test that change is in test and no test that testName
ends with Test
like isCheckstyleModuleTest
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These 2 convert methods (convertModuleTestChangeToModuleFullName
and convertModuleChangeToExtractInfo
) make me feel like you are trying to put too much into this class that may not be best as a utility method. They are so reliant on external factors on other tests done before this method is called.
Should we make convertJavaSourcePathToFullName
public and have the main component do the work instead of these methods?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The main concern is NAME_TO_MODULE_EXTRACT_INFO
map. It should be placed in ModuleUtils
since isCheckstyleModule
, isCheckstyleUtility
, isCheckstyleModuleTest
all need it.
convertModuleTestChangeToModuleFullName
is OK to move out from ModuleUtils
now. But convertModuleChangeToExtractInfo
could not, since we need access NAME_TO_MODULE_EXTRACT_INFO
to get the converted info. I don't think we should place NAME_TO_MODULE_EXTRACT_INFO
somewhere else.
My opinion now is: keeping these two methods here, then adding a if-condition at the beginning of each, throwing a IllegalArgumentException
if the parameter is not valid(not checkstyle module, or not checkstyle module test)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These methods just seem too specific to be an actual utility method. 1 is unused, so it could be one issue for why I can't see a good way to fix it right now.
Yes, the map should stay.
I was thinking not to move them out but having simplier methods like getModuleInfo
(take full module name as string parameter) so you don't have to move it out. The only issue for the test method would be TEST_POSTFIX
which I don't see how you can get around for now.
Otherwise, you can throw the exception, but the test one also needs to make sure it ends with Test
.
Usage of this method should never throw the exception, so you have to be real careful in it's use. I might make an issue to look at it later when we implement more in the module component.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was thinking not to move them out but having simplier methods like getModuleInfo (take full module name as string parameter)
I am fine with this.
Since actually we haven't use convertModuleTestChangeToModuleFullName
now. How about just remove it now and decide it later when we have more codes of handling test files later. Maybe it would be clearer of how to handle this at that time.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, just remove it than.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
*/ | ||
public static List<ModuleInfo> generate(List<GitChange> changes) { | ||
final Map<String, ModifiableModuleInfo> moduleInfos = new LinkedHashMap<>(); | ||
for (GitChange change : changes) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
empty line before for
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
* @param path the path of Java source file | ||
* @return the corresponding full qualified name | ||
*/ | ||
private static String convertJavaSourcePathToFullName(String path) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
convertModuleChangeToExtractInfo
and convertModuleTestChangeToModuleFullName
don't really need GitChange
but you leave convertJavaSourcePathToFullName
as String
even though everywhere that uses it has GitChange
available.
Change this parameter to GitChange.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
private static final String TEST_POSTFIX = "Test"; | ||
|
||
/** The map of full qualified name to module extract info. */ | ||
private static final Map<String, ModuleExtractInfo> NAME_TO_MODULE_EXTRAT_INFO = |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
EXTRAT
=> EXTRACT
Check for this misspelling everywhere.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
57c2f7b
to
5a0a1b1
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Final thing.
* @return true if the corresponding file of a change may be considered as | ||
* a Java test source file. | ||
*/ | ||
private static boolean isJavaTestSource(GitChange change) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One last thing, please group isJava...
methods together. Do the same for isCheckstyle...
methods.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
b6ff965
to
0be132c
Compare
Merged. |
#19
This PR contains many things. Some changes and discussions may be better in other issue/PR. Free feel to point them out and create new issues for them.
ModuleInfo
would contains many fields). So I think it is a good choice to introduce such library. FYI: libraries with similar function: AutoValue, FreeBuilderPackageObjectFactory
and write a code snippet inPackageObjectFactoryTest
to generate it. Contains the following info: name, parent, packageName, hierarchies, interfaces. But hierarchies, interfaces haven't been used in code yet.pom.xml
, splitting main and test dependencies. Maybe it is better to do that in a separate PR.pom.xml
was changed a bit to exclude the generated files from coverage check.