Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Krystal 9 Base work #327

Merged
merged 49 commits into from
Feb 5, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
955f1d1
Add initial support for facet ids and resolver ids
RamAnvesh Apr 22, 2024
176b920
Refactor and fix unit tests
RamAnvesh Apr 25, 2024
4b03167
Bump up version
RamAnvesh Apr 25, 2024
b60a9cf
Add initial support for new model code gen
RamAnvesh Apr 30, 2024
c475a56
Finish new model codegen for request and simple facets. Batch and Imp…
RamAnvesh May 6, 2024
c903fcd
Start facet id compatibility in vajram impls
RamAnvesh May 7, 2024
8debaf3
Correct canFanout method impl
RamAnvesh May 10, 2024
b9dfb73
Resolve merge conflicts
RamAnvesh May 14, 2024
61d8247
Merge branch 'krystal7' into facet_ids
RamAnvesh May 16, 2024
a0af434
Resolving merge conflicts
RamAnvesh May 16, 2024
551fbb4
Resolving merge conflicts
RamAnvesh May 20, 2024
a71a5ef
Resolving merge conflicts
RamAnvesh May 20, 2024
d0dc73a
Make message digest nonNull
RamAnvesh May 20, 2024
965db95
Resolve merge conflicts
RamAnvesh May 20, 2024
bba0143
Merge branch 'main' into facet_ids
RamAnvesh May 20, 2024
552bcc3
Added class diagrams - Old, Newly implemented, Proposed
May 29, 2024
58b2cc1
Added implementation for generating Batch and Common interfaces for B…
May 29, 2024
cb65d19
Fixed compilation of tests
May 29, 2024
469b97e
Some more corrections
Jun 4, 2024
e6e565c
added sample batch classes
Jul 7, 2024
6f0a61b
changed codegen logic to nest batch and common facet interfaces and c…
Jul 7, 2024
814cfdb
Added logic to get facets in outpput logic based on convention.
Oct 29, 2024
bee7f3d
Merge facet_ids into facet_ids_restructure
RamAnvesh Dec 5, 2024
5adbca5
Merge pull request #305 from RamAnvesh/facet_ids_restructure
RamAnvesh Dec 5, 2024
ad1dfa6
Correct recursive method codegen
RamAnvesh Dec 5, 2024
59a03ee
Resolved merge conflicts
RamAnvesh Dec 10, 2024
6fe17bf
Fix a few test cases
RamAnvesh Dec 10, 2024
09d7e8a
Add resolver generation code
RamAnvesh Dec 16, 2024
34ff99d
Fix nullchecker errors
RamAnvesh Dec 16, 2024
aed1ac0
Resolve merge conflicts
RamAnvesh Dec 16, 2024
d60f1f6
Initial support of facet getter and setter lambdas
RamAnvesh Dec 18, 2024
b267d45
Merge branch 'main' into facet_ids_restructure
RamAnvesh Dec 25, 2024
c88593b
Formalize facet specs
RamAnvesh Jan 21, 2025
4cd6aed
Fix bug where duplicate values or empty list or null list passed to M…
RamAnvesh Jan 22, 2025
a84121c
Remove deleted vajram from Vajrams scope file. Bump up version
RamAnvesh Jan 22, 2025
0de8b25
Merge pull request #321 from RamAnvesh/v8.x
RamAnvesh Jan 22, 2025
d05584f
Resolve merge conflicts
RamAnvesh Jan 22, 2025
58ef35c
Resolve merge conflicts
RamAnvesh Jan 22, 2025
b41639e
Continue facet spec standardization and update code gen accordingly
RamAnvesh Jan 30, 2025
ce50f14
Fix some test cases
RamAnvesh Jan 30, 2025
7337da8
Fix more test cases
RamAnvesh Jan 31, 2025
a900c7e
Fix all test cases
RamAnvesh Feb 1, 2025
4f70d81
Full build and code cleanup
RamAnvesh Feb 2, 2025
7cb08f5
Add support for Mandatory annotatation with ifNotSet strategies. Make…
RamAnvesh Feb 4, 2025
2f1c751
Upgrade java code standard plugin
RamAnvesh Feb 4, 2025
f33a364
[javaCodeStd] disable checks for test code and generated code. Bump u…
RamAnvesh Feb 4, 2025
c4927d7
Upgrade code standard plugin. Fix warnings in krystal-common
RamAnvesh Feb 4, 2025
899c1bf
Code cleanup
RamAnvesh Feb 5, 2025
660d87c
Standardize FacetValues class name and suppress error prone warnings …
RamAnvesh Feb 5, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
1,819 changes: 808 additions & 1,011 deletions .idea/inspectionProfiles/Project_Default.xml

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions .idea/misc.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ Single Bill-Of-Materials artefact with all krystal artefacts versions defined.
independant and agnostic of each other. Classes common to both vajram and krystex modules are
in `krystal-common` module.
6. `com.flipkart.krystal:vajram-guice`: This module contains a kryon decorator which allows
the `@Inject` facets in vajrams to be provided by the guice injector. See `vajram-samples` module
the `@Inject` facetValues in vajrams to be provided by the guice injector. See `vajram-samples` module
for how to use this decorator.

## Preface
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
00:00:01,000 --> 00:00:02,000
<font size="10px">
Krystal receives a request to execute Vajram V1.
The request contains the values of all the inputDef facets of V1.
The request contains the values of all the facetDef facets of V1.
V1 is ready for execution.
</font>

Expand All @@ -11,8 +11,8 @@ V1 is ready for execution.
<font size="10px">
V1 depends on vajrams V2, V3, and V4 in sequential order,
i.e V3 waits for V2 and V4 waits for V3.
V1 constructs the request for V2 containing values for V2's inputDef facets.
This is called inputDef resolution.
V1 constructs the request for V2 containing values for V2's facetDef facets.
This is called facetDef resolution.
V2 is ready for execution.
</font>

Expand Down Expand Up @@ -142,7 +142,7 @@ V8 succeeds.
00:00:40,000 --> 00:00:42,000
<font size="10px">
V8's response is intercepted by the caching decorator.
The response is cached with the inputDef facets values of V8 as key.
The response is cached with the facetDef facets values of V8 as key.
</font>

22
Expand Down
44 changes: 23 additions & 21 deletions build.gradle
Original file line number Diff line number Diff line change
@@ -1,17 +1,13 @@
plugins {
id 'com.flipkart.java.code.standard' version '0.1.2-SNAPSHOT'
id 'com.flipkart.java.code.standard' version '3.5'
}

ext {
clojarsusername = project.properties['clojarsusername'] ?: ""
clojarspassword = project.properties['clojarspassword'] ?: ""
krystal_version = '9.0.0-SNAPSHOT'
}

checkerFramework {
extraJavacArgs = [
"-Astubs=${rootDir}/config/checker/stubs",
]
//Use this during development time to fasten compilation by disabling heavy compiler plugins
fast_compile = false
}

subprojects {
Expand Down Expand Up @@ -44,14 +40,14 @@ subprojects {
}
}

configure(subprojects.findAll { it.name != 'krystal-bom' }) {
configure(subprojects.findAll { it.name != 'krystal-bom' && it.name != "code-coverage-report" }) {
apply plugin: 'java-library'
apply plugin: 'com.flipkart.java.code.standard'
apply plugin: 'jacoco'
if (!fast_compile) {
apply plugin: 'com.flipkart.java.code.standard'
}

dependencies {
implementation 'com.google.guava:guava'
implementation 'org.checkerframework:checker-qual'
implementation 'org.slf4j:slf4j-api'
implementation 'jakarta.inject:jakarta.inject-api'

Expand Down Expand Up @@ -82,7 +78,6 @@ configure(subprojects.findAll { it.name != 'krystal-bom' }) {
implementation 'org.yaml:snakeyaml:2.2'

implementation 'com.google.guava:guava:33.0.0-jre'
implementation 'org.checkerframework:checker-qual:3.42.0'
implementation 'org.slf4j:slf4j-api:2.0.11'
implementation 'jakarta.inject:jakarta.inject-api:2.0.1'
implementation 'org.reflections:reflections:0.10.2'
Expand All @@ -95,30 +90,37 @@ configure(subprojects.findAll { it.name != 'krystal-bom' }) {
annotationProcessor 'com.flipkart.krystal:vajram-codegen:' + krystal_version
testAnnotationProcessor 'com.flipkart.krystal:vajram-codegen:' + krystal_version

String lombok_version = '1.18.30'
String lombok_version = '1.18.36'
implementation 'org.projectlombok:lombok:' + lombok_version
annotationProcessor 'org.projectlombok:lombok:' + lombok_version
testAnnotationProcessor 'org.projectlombok:lombok:' + lombok_version
}
}

checkerFramework {
extraJavacArgs = ["-Astubs=${rootDir}/config/checker/stubs",]
excludeTests = true
}

jacoco {
toolVersion = "0.8.11"
if (!fast_compile) {
checkerFramework {
excludeTests = true
}
}

java {
withSourcesJar()
}

tasks.withType(JavaCompile).configureEach {
compileJava {
options.compilerArgs += '-Xlint:unchecked'
options.deprecation = true
}

tasks.named('sourcesJar').configure { mustRunAfter('compileJava') }
}

gradle.taskGraph.whenReady { taskGraph ->
println "======================================="
println "Tasks"
println "======================================="
taskGraph.getAllTasks().eachWithIndex { task, n ->
println "${n + 1} $task"
task.dependsOn.eachWithIndex { depObj, m -> println " ${m + 1} $depObj" }
}
}
47 changes: 47 additions & 0 deletions docs/Krystal_AutoGenerated_Models.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
## Scope

This document describes the models that are auto-generated by Krystal and their function and design.

When a developer writes a vajram, the Vajram code generation annotation processor generates a few
classes which model the facetValues of the vajram. These models are of these types:

* Request
* This object contains only and all [
`@Input`](../vajram/vajram-java-sdk/src/main/java/com/flipkart/krystal/vajram/facets/Input.java)
facetValues of the vajram. It is intended to be usable as an independant object by application
developers (Ex: when
invoking a vajram using the [
`KrystalVajramExecutor`](../vajram/vajram-krystex/src/main/java/com/flipkart/krystal/vajramexecutor/krystex/KrystexVajramExecutor.java)
or in some complex input resolvers which set inputs into Request Builders)
* Facets
* This object contains all the Facets of the vajram.
* It is not intended to be used by application developers directly. Instead it acts as a
state holder for the facetValues values as
they are computed during the lifecycle of the vajram's execution.
* In older versions of Krystal, this was done using simple Maps which contained the facet name
as key and facet value
as the value. Since Krystal 9, this facetValues class is generated and used instead as getting and
setting fields in objects takes lesser CPU cycles and is lighter on memory footprint.
* Batch Facets (Only applicable if batching is enabled)
* This object contains values of all facets which have been configured to be batched together
across vajram invocations, i.e. annotated with [
`@Batch(forOutputLogic = true)`](../vajram/vajram-java-sdk/src/main/java/com/flipkart/krystal/vajram/batching/Batch.java)
* Unlike the other the Request and Facets classes which primiarily used for communication across
vajrams (maybe over the wire), persistance of facet values (in case of async orchestration) or
for internal platform use, the Batch Facets class is excusilvely generated so that the
application developer can access all the Batched facets which have been marked as
`@Batch(forOutputLogic = true)`
* Common Facets (Only applicable if batching is enabled)
* This object contains all facetValues which are not tagged with `@Batch`. These are those facetValues
which remain same across the various batches. In Query Language terminology they are analogous
the "group by" operation.

## Class Design

Each of these four categories of objects actually correspond to 3 interfaces:

* An Interface
* An Immuatable final class
* An Mutable builder for the Immutable class

This design allows code to
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
package com.flipkart.krystal.vajram.guice;

import static com.flipkart.krystal.data.Errable.errableFrom;
import static com.flipkart.krystal.data.Errable.nil;
import static com.flipkart.krystal.facets.FacetType.INJECTION;

import com.flipkart.krystal.data.Errable;
import com.flipkart.krystal.except.StackTracelessException;
import com.flipkart.krystal.vajram.VajramID;
import com.flipkart.krystal.vajram.facets.InputDef;
import com.flipkart.krystal.vajram.facets.InputSource;
import com.flipkart.krystal.vajram.facets.specs.FacetSpec;
import com.flipkart.krystal.vajramexecutor.krystex.inputinjection.VajramInjectionProvider;
import com.google.inject.Injector;
import com.google.inject.Key;
Expand All @@ -18,6 +19,7 @@
import java.util.List;
import java.util.Map;
import java.util.Optional;
import lombok.NonNull;
import lombok.extern.slf4j.Slf4j;

@Slf4j
Expand All @@ -31,9 +33,9 @@ public VajramGuiceInjector(Injector injector) {
}

@Override
public <T> Errable<T> get(VajramID vajramID, InputDef<T> inputDef) {
if (!inputDef.sources().contains(InputSource.SESSION)) {
return Errable.empty();
public <T> Errable<@NonNull T> get(VajramID vajramID, FacetSpec<T, ?> facetDef) {
if (!facetDef.facetTypes().contains(INJECTION)) {
return nil();
}
return errableFrom(
() -> {
Expand All @@ -43,11 +45,11 @@ public <T> Errable<T> get(VajramID vajramID, InputDef<T> inputDef) {
providerCache
.computeIfAbsent(vajramID, _v -> new LinkedHashMap<>())
.computeIfAbsent(
inputDef.name(),
facetDef.name(),
_i -> {
try {
Type type = inputDef.type().javaReflectType();
var annotation = getQualifier(vajramID, inputDef);
Type type = facetDef.type().javaReflectType();
var annotation = getQualifier(vajramID, facetDef);
if (annotation.isEmpty()) {
return injector.getProvider(Key.get(type));
} else {
Expand All @@ -62,9 +64,9 @@ public <T> Errable<T> get(VajramID vajramID, InputDef<T> inputDef) {
});
}

private Optional<Annotation> getQualifier(VajramID vajramID, InputDef<?> inputDef) {
private <T> Optional<Annotation> getQualifier(VajramID vajramID, FacetSpec<T, ?> facetDef) {
List<Annotation> qualifierAnnotations =
inputDef.tags().annotations().stream()
facetDef.tags().annotations().stream()
.<Annotation>mapMulti(
(tag, consumer) -> {
boolean isQualifierAnno =
Expand All @@ -82,7 +84,7 @@ private Optional<Annotation> getQualifier(VajramID vajramID, InputDef<?> inputDe
throw new IllegalStateException(
("More than one @jakarta.inject.Qualifier annotations (%s) found on input '%s' of vajram '%s'."
+ " This is not allowed")
.formatted(qualifierAnnotations, inputDef.name(), vajramID.vajramId()));
.formatted(qualifierAnnotations, facetDef.name(), vajramID.vajramId()));
}
}
}
2 changes: 1 addition & 1 deletion fk-java-code-std-plugin/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ plugins {
}

group 'com.flipkart.java.code.standard'
version '3.4'
version '3.5'

gradlePlugin {
plugins {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ import com.diffplug.gradle.spotless.SpotlessExtension
import org.checkerframework.gradle.plugin.CheckerFrameworkExtension
import org.gradle.api.Plugin
import org.gradle.api.Project
import org.gradle.api.Task
import org.gradle.api.tasks.compile.JavaCompile
import org.gradle.testing.jacoco.plugins.JacocoPluginExtension

class FkJavaCodeStandard implements Plugin<Project> {
Expand Down Expand Up @@ -85,6 +87,10 @@ class FkJavaCodeStandard implements Plugin<Project> {
project.pluginManager.apply('net.ltgt.errorprone')

project.dependencies.add('errorprone', 'com.google.errorprone:error_prone_core:2.27.1')
project.tasks.compileTestJava.configure { JavaCompile task -> task.options.errorprone.enabled = false }
project.tasks.withType(JavaCompile).configureEach { JavaCompile task ->
task.options.errorprone.disableWarningsInGeneratedCode = true
}
}

private static void junitPlatform(Project project) {
Expand Down
2 changes: 1 addition & 1 deletion honeycomb/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ asynchronous workflows involving hundreds of steps. The core features supported
1. Understanding individual fields and nested fields in the workflow payload.
1. Tracking workflow instance progress at individual field level in the payload.
1. Support fork-join where one workflow forks into N instances of another workflow and then collects
the results from all of them.
the depResponses from all of them.
1. Maintain a call-hierarchy of parent to forked workflow instance.
1. Manage queueing and asynchronous communications among all asynchronous portions of the workflow.
1. Support observability by providing visual interfaces to the following data
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
package com.flipkart.krystal.data;

public sealed interface DepResponse<R extends Request<T>, T> extends FacetValue<T>
permits FanoutDepResponses, One2OneDepResponse {}
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
package com.flipkart.krystal.data;

import java.util.Optional;
import org.checkerframework.checker.nullness.qual.Nullable;

public interface DividerRequest {
public interface Option1 {
int numerator();

Optional<Integer> denominator();
}

public interface Option2 {

Optional<Integer> numerator();

Optional<Integer> denominator();
}

public interface Option3 {
int numerator();

int denominator_OrDefault();
}

public interface Option4 {
@Nullable Integer numerator();

@Nullable Integer denominator();
}

public interface Option5 {
int numerator();

@Nullable Integer denominator();
}
}
Loading