Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IFDS core refactor #225

Open
wants to merge 64 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
97cf2a2
[ifds] feat: initial ifds refactor
sergeypospelov Mar 26, 2024
5e9b7cf
[ifds] fix: function to property
sergeypospelov Mar 28, 2024
6edef8f
[ifds] fix: useless class
sergeypospelov Mar 28, 2024
9f69164
[ifds] style: renamings
sergeypospelov Mar 28, 2024
dd8e2a7
[ifds] fix: remove `Message` interface
sergeypospelov Mar 29, 2024
979e956
[ifds] fix: remove `AckMessage` interface
sergeypospelov Mar 29, 2024
3880b3b
[ifds] fix: `ack` -> `ask`
sergeypospelov Mar 29, 2024
7e765d4
[ifds] feat: reasons, `runnerId` property, `IfdsResult`
sergeypospelov Apr 4, 2024
e4138c0
[ifds] fix: revert `signPublication`
sergeypospelov Apr 5, 2024
383f41d
[ifds] feat: unused analyzer
sergeypospelov Apr 5, 2024
bef6289
[ifds] feat: multi chunk + tests
sergeypospelov Apr 9, 2024
f8146fd
[ifds] feat: graceful termination
sergeypospelov Apr 15, 2024
534743a
[ifds] chore: removed generic fact parameter
sergeypospelov Apr 15, 2024
72cffcb
[ifds] fix: IRTest and Type Tests
sergeypospelov Apr 15, 2024
d4c5e35
[ifds] fix: build-and-test.yml single run
sergeypospelov Apr 15, 2024
6983a95
[ifds] fix: build-and-test.yml workflow_dispatch
sergeypospelov Apr 15, 2024
1ddcba9
[ifds] fix: remove foojay
sergeypospelov Apr 15, 2024
75fb2da
[ifds] chore: `Channel` to `CompletableDeferred`
sergeypospelov Apr 16, 2024
6b5a78c
[ifds] chore: cleanup/format
Lipen Apr 22, 2024
0ebc6ee
[ifds] fix: use kotlin("test")
Lipen Apr 22, 2024
5eb5da3
[ifds] chore: import alias
Lipen Apr 22, 2024
f30cff3
[ifds] chore: add todo about renaming systemOf to system
Lipen Apr 22, 2024
97ee4fb
[ifds] fix: use TaintAnalyzer
Lipen Apr 22, 2024
8c068af
[ifds] chore: format
Lipen Apr 22, 2024
af379f2
[ifds] chore: make collectNpeResults similar to collectTaintResults
Lipen Apr 22, 2024
411d7f6
[ifds] fix: cancel "stop-after-timeout" job
Lipen Apr 22, 2024
e7abe19
[ifds] fix: resume after await
Lipen Apr 22, 2024
1d978d5
[ifds] fix: review comments
sergeypospelov May 6, 2024
4eeaeca
[ifds] wip: remove some old ifds related code
sergeypospelov May 13, 2024
f381663
[ifds] wip: remove Reason
sergeypospelov May 14, 2024
cc25fe5
[ifds] wip: remove old Edge & Vertex, rename stmt -> statement
sergeypospelov May 14, 2024
6e53fe7
[ifds] wip: rename JcFlowScope
sergeypospelov May 14, 2024
788fbd1
[ifds] wip: draft
sergeypospelov May 14, 2024
b2862b1
[ifds] wip: change flow functions interface
sergeypospelov May 14, 2024
69dcfd0
[ifds] wip: change flow functions interface, renamings
sergeypospelov May 14, 2024
3b7431e
[ifds] wip: renamings
sergeypospelov May 14, 2024
ae528ee
[ifds] wip: renamings
sergeypospelov May 14, 2024
36d5c3c
[ifds] wip: get rid of custom events
sergeypospelov May 14, 2024
554d8cf
[ifds] wip: fix CLI
sergeypospelov May 14, 2024
753d747
[ifds] wip: fix gradle
sergeypospelov May 14, 2024
cb1ddbb
[ifds] wip: fix test
sergeypospelov May 14, 2024
36c3cf8
[ifds] wip: remove entities, simplify code
sergeypospelov May 14, 2024
69b8b88
[ifds] wip: start analysis, unify extension functions, builders
sergeypospelov May 14, 2024
b69f271
[ifds] wip: remove typealiases
sergeypospelov May 14, 2024
5ded81b
[ifds] wip: fix tests
sergeypospelov May 15, 2024
3f153a3
[ifds] wip: refactored indirection handler a bit
sergeypospelov May 15, 2024
d237955
[ifds] wip: closing actor system
sergeypospelov May 15, 2024
08e4def
[ifds] wip: restructure packages
sergeypospelov May 15, 2024
e1f2e0e
[ifds] wip: graph factory
sergeypospelov May 15, 2024
1591d8b
[ifds] wip: remove graph dependency from flow functions, add NoResolv…
sergeypospelov May 15, 2024
745b509
[ifds] wip: remove foojay again!
sergeypospelov May 15, 2024
30a2753
[ifds] wip: CallAction
sergeypospelov May 16, 2024
9040154
[ifds] wip: CallAction comments
sergeypospelov May 16, 2024
c433868
[ifds] wip: add some tests
sergeypospelov May 16, 2024
d213980
[ifds] wip: minor
sergeypospelov May 16, 2024
5d489fe
[ifds] wip: exception
sergeypospelov May 16, 2024
5992089
[ifds] wip: add statistics every 10 seconds + options
sergeypospelov May 16, 2024
a56f4a2
Cleanup
Lipen May 17, 2024
194e21b
Reorder nested loops
Lipen May 17, 2024
526abc4
Improve logging messages
Lipen May 17, 2024
320641d
Use hashMapOf
Lipen May 17, 2024
ab75c44
Fix typo in user actor name
Lipen May 17, 2024
05a1e00
Import
Lipen May 17, 2024
1b5872c
Support "unknown chunks" (null)
Lipen May 17, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/build-and-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,12 @@ name: Build and run tests

on:
push:
branches:
- develop
pull_request:
branches:
- develop
workflow_dispatch:

permissions:
contents: read
Expand Down
2 changes: 1 addition & 1 deletion buildSrc/src/main/kotlin/Dependencies.kt
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ object Versions {
const val jooq = "3.14.16"
const val juliet = "1.3.2"
const val junit = "5.9.2"
const val kotlin = "1.7.21"
const val kotlin = "1.9.0"
const val kotlin_logging = "1.8.3"
const val kotlinx_benchmark = "0.4.4"
const val kotlinx_cli = "0.3.5"
Expand Down
2 changes: 2 additions & 0 deletions jacodb-analysis/README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
# OUTDATED!

# Module `jacodb-analysis`

The `jacodb-analysis` module allows launching application dataflow analyses.
Expand Down
8 changes: 8 additions & 0 deletions jacodb-analysis/actors/build.gradle.kts
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
dependencies {
implementation(Libs.kotlin_logging)
implementation(Libs.slf4j_simple)
implementation(Libs.kotlinx_coroutines_core)

testImplementation(kotlin("test"))
testImplementation(Libs.mockk)
}
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,15 @@
* limitations under the License.
*/

package org.jacodb.analysis.taint
package org.jacodb.actors.api

import org.jacodb.analysis.ifds.Edge
import org.jacodb.analysis.ifds.Runner
import org.jacodb.analysis.ifds.Vertex
import org.jacodb.actors.api.signal.Signal

typealias TaintVertex = Vertex<TaintDomainFact>
typealias TaintEdge = Edge<TaintDomainFact>
typealias TaintRunner = Runner<TaintDomainFact>
interface Actor<M> {
suspend fun receive(message: M)
suspend fun receive(signal: Signal) {
if (signal is Signal.Exception) {
signal.exception.printStackTrace()

Check warning on line 25 in jacodb-analysis/actors/src/main/kotlin/org/jacodb/actors/api/Actor.kt

View check run for this annotation

Codecov / codecov/patch

jacodb-analysis/actors/src/main/kotlin/org/jacodb/actors/api/Actor.kt#L25

Added line #L25 was not covered by tests
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
/*
* Copyright 2022 UnitTestBot contributors (utbot.org)
* <p>
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.jacodb.actors.api

import mu.KLogger
import kotlin.time.Duration

interface ActorContext<M> : ActorSpawner {
val self: ActorRef<M>

suspend fun <TargetMessage> ActorRef<TargetMessage>.send(message: TargetMessage)

suspend fun sendSelfWithDelay(message: M, waitDelay: Duration)

fun stop()
fun resume()

val logger: KLogger
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
/*
* Copyright 2022 UnitTestBot contributors (utbot.org)
* <p>
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.jacodb.actors.api

fun interface ActorFactory<M> {
fun ActorContext<M>.create(): Actor<M>
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
/*
* Copyright 2022 UnitTestBot contributors (utbot.org)
* <p>
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.jacodb.actors.api

interface ActorPath {
operator fun div(name: String): ActorPath
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
/*
* Copyright 2022 UnitTestBot contributors (utbot.org)
* <p>
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.jacodb.actors.api

// Abstract class used instead of interface to support internal abstract functions
abstract class ActorRef<in M>(
val path: ActorPath,
) {
internal abstract suspend fun receive(message: M): Boolean
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
/*
* Copyright 2022 UnitTestBot contributors (utbot.org)
* <p>
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.jacodb.actors.api

import org.jacodb.actors.api.options.SpawnOptions

interface ActorSpawner {
fun <ChildMessage> spawn(
name: String,
options: SpawnOptions = SpawnOptions.default,
actorFactory: ActorFactory<ChildMessage>,
): ActorRef<ChildMessage>

fun child(name: String): ActorRef<*>?
fun children(): Map<String, ActorRef<*>>

fun stopChild(name: String)
fun resumeChild(name: String)
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
/*
* Copyright 2022 UnitTestBot contributors (utbot.org)
* <p>
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.jacodb.actors.api

enum class ActorStatus {
IDLE,
BUSY
}
Original file line number Diff line number Diff line change
Expand Up @@ -14,18 +14,23 @@
* limitations under the License.
*/

package org.jacodb.analysis.sarif
package org.jacodb.actors.api

import io.github.detekt.sarif4k.Level
import org.jacodb.analysis.ifds.TraceGraph
import kotlinx.coroutines.CompletableDeferred
import mu.KLogger

data class VulnerabilityInstance<Fact>(
val traceGraph: TraceGraph<Fact>,
val description: VulnerabilityDescription,
)
interface ActorSystem<Message> : AutoCloseable {
val name: String

data class VulnerabilityDescription(
val ruleId: String?,
val message: String?,
val level: Level = Level.Warning,
)
suspend fun send(message: Message)

suspend fun <R> ask(messageBuilder: (CompletableDeferred<R>) -> Message): R

suspend fun awaitCompletion()

suspend fun resume()

fun stop()

val logger: KLogger
}
Original file line number Diff line number Diff line change
Expand Up @@ -14,20 +14,24 @@
* limitations under the License.
*/

package org.jacodb.analysis.graph
package org.jacodb.actors.api.options

import org.jacodb.api.cfg.JcExpr
import org.jacodb.api.cfg.JcInst
import org.jacodb.api.cfg.JcInstLocation
import org.jacodb.api.cfg.JcInstVisitor
import kotlinx.coroutines.channels.Channel

data class JcNoopInst(override val location: JcInstLocation) : JcInst {
override val operands: List<JcExpr>
get() = emptyList()
fun interface ChannelFactory {
fun create(): Channel<Any?>

override fun <T> accept(visitor: JcInstVisitor<T>): T {
return visitor.visitExternalJcInst(this)
}
companion object {
fun unlimited() = ChannelFactory {
Channel(capacity = Channel.UNLIMITED)
}

fun rendezvous() = ChannelFactory {
Channel(capacity = Channel.RENDEZVOUS)
}

Check warning on line 31 in jacodb-analysis/actors/src/main/kotlin/org/jacodb/actors/api/options/ChannelFactory.kt

View check run for this annotation

Codecov / codecov/patch

jacodb-analysis/actors/src/main/kotlin/org/jacodb/actors/api/options/ChannelFactory.kt#L29-L31

Added lines #L29 - L31 were not covered by tests

override fun toString(): String = "noop"
fun buffered(size: Int) = ChannelFactory {
Channel(capacity = size)
}

Check warning on line 35 in jacodb-analysis/actors/src/main/kotlin/org/jacodb/actors/api/options/ChannelFactory.kt

View check run for this annotation

Codecov / codecov/patch

jacodb-analysis/actors/src/main/kotlin/org/jacodb/actors/api/options/ChannelFactory.kt#L33-L35

Added lines #L33 - L35 were not covered by tests
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -14,20 +14,24 @@
* limitations under the License.
*/

package org.jacodb.analysis.taint
package org.jacodb.actors.api.options

import org.jacodb.analysis.ifds.TraceGraph
import org.jacodb.analysis.sarif.VulnerabilityDescription
import org.jacodb.analysis.sarif.VulnerabilityInstance
import kotlinx.coroutines.channels.Channel

fun TaintVulnerability.toSarif(
graph: TraceGraph<TaintDomainFact>,
): VulnerabilityInstance<TaintDomainFact> {
return VulnerabilityInstance(
graph,
VulnerabilityDescription(
ruleId = null,
message = rule?.ruleNote
)
class SpawnOptions(
val channelFactory: ChannelFactory,
) {
fun channelFactory(channelFactory: ChannelFactory) = SpawnOptions(
channelFactory = channelFactory,
)

Check warning on line 26 in jacodb-analysis/actors/src/main/kotlin/org/jacodb/actors/api/options/SpawnOptions.kt

View check run for this annotation

Codecov / codecov/patch

jacodb-analysis/actors/src/main/kotlin/org/jacodb/actors/api/options/SpawnOptions.kt#L24-L26

Added lines #L24 - L26 were not covered by tests

fun channel(channel: Channel<Any?>) = SpawnOptions(
channelFactory = { channel },

Check warning on line 29 in jacodb-analysis/actors/src/main/kotlin/org/jacodb/actors/api/options/SpawnOptions.kt

View check run for this annotation

Codecov / codecov/patch

jacodb-analysis/actors/src/main/kotlin/org/jacodb/actors/api/options/SpawnOptions.kt#L28-L29

Added lines #L28 - L29 were not covered by tests
)

companion object {
val default = SpawnOptions(
ChannelFactory.unlimited(),
)
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
/*
* Copyright 2022 UnitTestBot contributors (utbot.org)
* <p>
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.jacodb.actors.api.signal

sealed interface Signal {
data object Start : Signal
data object PostStop : Signal
data class Exception(val exception: java.lang.Exception) : Signal

Check warning on line 22 in jacodb-analysis/actors/src/main/kotlin/org/jacodb/actors/api/signal/Signal.kt

View check run for this annotation

Codecov / codecov/patch

jacodb-analysis/actors/src/main/kotlin/org/jacodb/actors/api/signal/Signal.kt#L22

Added line #L22 was not covered by tests
}
Loading
Loading