Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rust support #250

Merged
merged 173 commits into from
Sep 7, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
173 commits
Select commit Hold shift + click to select a range
7d4769d
kaitai + rust = latest development from bspeice + rebase + small changes
Agile86 Jul 6, 2022
be78dcc
handle strings in ASCII, UTF-8, UTF-16 encodings.
Agile86 Jul 6, 2022
a5e8066
instance generation fixed.
Agile86 Jul 6, 2022
0768146
decode almost any string encoding.
Agile86 Jul 7, 2022
20fbd40
handle string literals.
Agile86 Jul 11, 2022
a61807a
BytesType => String instead of [u8]
revitalyr Jul 11, 2022
1a63ea8
updated KResult type.
Agile86 Jul 13, 2022
dddf0dc
BytesType => Vec<u8>, remove lifetime
revitalyr Jul 13, 2022
3706673
Merge branch 'rust_basic_support' into rust_tuning
Jul 13, 2022
0c7c0b2
debug output added.
Agile86 Jul 13, 2022
ff45079
implementing UserTypeFromBytes
revitalyr Jul 13, 2022
585973c
clippy doesn't like borrowing of io.read_bytes(...), use allocateIO(...)
revitalyr Jul 14, 2022
ba4a3c2
SwitchType implemented.
Agile86 Jul 17, 2022
b93ede1
cast type into directly (help with enums).
Agile86 Jul 18, 2022
4a31b6c
kaitaiPrimitiveToNativeType(...) processes EnumType (not checked yet)
revitalyr Jul 18, 2022
694f343
Merge branch 'rust_basic_support' of https://github.com/Agile86/kaita…
revitalyr Jul 18, 2022
11fd4d1
generate helper function for SwitchEnum type.
Agile86 Jul 19, 2022
06148b5
condIfHeader implemented.
Agile86 Jul 20, 2022
f866710
generate SwitchEnum -> usize helper func only if enum is based on num…
Agile86 Jul 20, 2022
7dcf82e
restrict helper func from<SwitchEnum> into int for numeric types only.
Agile86 Jul 20, 2022
b61b4f7
correct reading enums, Option<TypedStack<Self::ParentStack>>
revitalyr Jul 20, 2022
3ebcf83
Switch implemented.
Agile86 Jul 21, 2022
5bcc7d6
repeat-eos fixed.
Agile86 Jul 21, 2022
98f0a0f
doInternalName
revitalyr Jul 21, 2022
42eb073
generate `from_file(path)`
revitalyr Jul 21, 2022
2b86db0
switch_manual_int_size.ksy implemented.
Agile86 Jul 22, 2022
052d61a
small bug fixed.
Agile86 Jul 22, 2022
e7f8567
handle UserTypeFromBytes[BytesEosType].
Agile86 Jul 22, 2022
407207b
read_bytes_term().into()
Agile86 Jul 22, 2022
0edd9c0
out full path of generated file
revitalyr Jul 22, 2022
d19d9dd
reformat `handleAssignmentSimple` for `UserType`
revitalyr Jul 22, 2022
d90a712
universalDoc implemented.
Agile86 Jul 23, 2022
a7b98d8
instances usage fixed.
Agile86 Jul 24, 2022
dba3a47
Merge branch 'rust_basic_support' of https://github.com/Agile86/kaita…
revitalyr Jul 24, 2022
66bd5db
attrProcess implemented.
Agile86 Jul 25, 2022
362a0f5
root generation value fixed.
Agile86 Jul 25, 2022
b481020
valid: & pos: implemented.
Agile86 Jul 25, 2022
ffed056
switchenum fixed handle same typename (in case of different endianness).
Agile86 Jul 27, 2022
5f301fd
Report details if NotImplementedError.
Agile86 Jul 28, 2022
c476a64
unnamed var generation and usage fixed.
Agile86 Jul 28, 2022
bb816c8
up to date
revitalyr Jul 30, 2022
2cbfd9e
Merge branch 'rust_basic_support' of https://github.com/Agile86/kaita…
revitalyr Jul 30, 2022
b857343
override `attrParse2(...)` to process `EnumType`
revitalyr Aug 1, 2022
948773e
correctly handle calling of instances.
Agile86 Aug 2, 2022
632a6ff
instances generation and usage reviewd.
Agile86 Aug 2, 2022
eb8d729
Merge remote-tracking branch 'origin/rust_tuning' into rust_basic_sup…
Agile86 Aug 2, 2022
bece5c1
useless code removed (base method is the same).
Agile86 Aug 2, 2022
9f63df1
process `BitsType`
revitalyr Aug 2, 2022
905073d
instances refactoring (using RefCell)
Agile86 Aug 5, 2022
0d5aa35
Clone added to workaround complex issues with instances calling other
Agile86 Aug 5, 2022
db4ed94
correctly handle min/max for float type.
Agile86 Aug 8, 2022
41defdd
Merge branch 'rust_basic_support' of https://github.com/Agile86/kaita…
revitalyr Aug 8, 2022
c35652e
handle _parent._parent.
Agile86 Aug 9, 2022
28a8f2a
doByteArrayNonLiteral implemented.
Agile86 Aug 9, 2022
0415b11
#[derive(..., Copy)] for enums
revitalyr Aug 9, 2022
ecd764c
no `&` in doEnumByLabel(...)
revitalyr Aug 9, 2022
598a179
Merge branch 'rust_basic_support' of https://github.com/Agile86/kaita…
revitalyr Aug 10, 2022
e37a99f
handle arrays correctly.
Agile86 Aug 11, 2022
087ac24
to_string from int fixed.
Agile86 Aug 11, 2022
c681bfc
remove Copy
revitalyr Aug 11, 2022
1d29b69
Merge branch 'rust_basic_support' of https://github.com/Agile86/kaita…
revitalyr Aug 11, 2022
426d3b2
add field `allClasses` to publish `classSpecs`
revitalyr Aug 11, 2022
7e39daa
add to `importList` all external types, remove some warnings
revitalyr Aug 11, 2022
2cd28b1
expr ref/deref finally fixed?
Agile86 Aug 12, 2022
395ebad
no parent case fixed.
Agile86 Aug 12, 2022
38d50db
Merge remote-tracking branch 'origin/rust_tuning' into rust_basic_sup…
Agile86 Aug 12, 2022
7ea6d39
enum ref fixed.
Agile86 Aug 12, 2022
974c4ab
`instanceCalculate` processes `EnumType`, add impl Default for $enumC…
revitalyr Aug 17, 2022
5a68cf2
`translate` for `EnumType` may use `try_from`
revitalyr Aug 17, 2022
b943048
default to not deref.
Agile86 Aug 18, 2022
ccfb6ac
Merge remote-tracking branch 'origin/rust_tuning' into rust_basic_sup…
Agile86 Aug 18, 2022
7968a80
`enumToInt` uses `i64::from` (enum_to_i.ksy)
revitalyr Aug 23, 2022
4844f52
`impl From<&$enumClass> for i64` (enum_to_i.ksy)
revitalyr Aug 23, 2022
528886e
handle params.
Agile86 Aug 24, 2022
3550542
cast to usize.
Agile86 Aug 25, 2022
4184139
.into() added.
Agile86 Aug 25, 2022
4cc25b5
`in_instance_need_deref_attr` -> `context_need_deref_attr`, use it in…
revitalyr Aug 25, 2022
e5efb6d
Merge branch 'rust_basic_support' of https://github.com/Agile86/kaita…
revitalyr Aug 26, 2022
d3e8ee6
use `super::` instead of `crate::`
revitalyr Aug 26, 2022
6fb9fe7
`doName` checks for param
revitalyr Aug 26, 2022
ac8ee1b
code restored.
Agile86 Aug 27, 2022
1653d52
strings ops, numeric ops.
Agile86 Sep 1, 2022
95ac649
int to enum.
Agile86 Sep 2, 2022
4f6daf5
strToInt with any base.
Agile86 Sep 2, 2022
0aca2f1
string compare fixed.
Agile86 Sep 3, 2022
f3384ff
repeatUntil initial version.
Agile86 Sep 4, 2022
2e76c36
always report stacktrace at exception.
Agile86 Sep 9, 2022
fbded32
condRepeatUntil implemented.
Agile86 Sep 9, 2022
2343761
instance and repeat until.
Agile86 Sep 15, 2022
2f71ce2
append RefCell to raw_ ids if they using in instance.
Agile86 Sep 16, 2022
bbe279a
root arg restored for instance methods.
Agile86 Sep 18, 2022
61d02fa
detect if in_instance.
Agile86 Sep 18, 2022
a7f5e16
handle RawIdentifier(InstanceIdentifier())
Agile86 Sep 22, 2022
888ce28
handle opaque types.
Agile86 Sep 24, 2022
7565497
getTopClass rewritted.
Agile86 Sep 24, 2022
a8a27b8
handle opaque types.
Agile86 Sep 24, 2022
0b31ec4
handle bitEndian flag.
Agile86 Oct 1, 2022
5ddee6d
handle endian.
Agile86 Oct 5, 2022
d75de69
enum into array fixed.
Agile86 Oct 6, 2022
6118dc8
small fix of numericOp.
Agile86 Oct 8, 2022
be05906
handle ProcessCustom.
Agile86 Oct 11, 2022
072746c
process_xor fixed.
Agile86 Oct 11, 2022
25b8a2f
closer behaviour for enums to C++ (enums to i64).
Agile86 Oct 12, 2022
08b367a
provide default parent if not provided (only for root).
Agile86 Oct 13, 2022
938b8a8
classToString implemented.
Agile86 Oct 21, 2022
792d349
parent type generation reviewed.
Agile86 Nov 15, 2022
30f4a99
EnumType need to clone.
Agile86 Nov 17, 2022
521f8ec
Merge remote-tracking branch 'origin/master' into rust_basic_support
Agile86 Nov 25, 2022
a7f48ae
always use buffered substreams.
Agile86 Nov 28, 2022
282ca92
RefCell for all. Root, parent as SharedType<>
Agile86 Dec 26, 2022
c83e131
Rust code generation updated.
Agile86 Dec 27, 2022
8dd7d1d
generate delegates for enums' variants
revitalyr Dec 27, 2022
711022d
`attributeReader` always returns `Ref<$typeName>` (`test_nav_parent_s…
revitalyr Dec 28, 2022
0514760
Merge branch 'rust_basic_support_v2' of https://github.com/Agile86/ka…
revitalyr Dec 29, 2022
040929f
use `cleanTypename` in generating delegates
revitalyr Dec 29, 2022
56205f3
generate delegates only for `UserType`
revitalyr Dec 30, 2022
61c2dcc
latest version of Rust code generation.
Agile86 Dec 31, 2022
a51092b
Merge branch 'rust_basic_support_v2' of https://github.com/Agile86/ka…
revitalyr Jan 2, 2023
6170f33
clean warnings
revitalyr Jan 3, 2023
c74309f
add `RefCell<...>` in `kaitaiTypeToNativeType`, remove `kaitaiTypeToN…
revitalyr Jan 3, 2023
fc1699d
'.as_ref().unwrap()' for `RefCell<Option<`
revitalyr Jan 3, 2023
37bcb02
parent access fixed + translate_args added.
Agile86 Jan 4, 2023
b09b820
need_deref extended with classspec + code cleanup.
Agile86 Jan 4, 2023
c938eff
Merge branch 'rust_basic_support_v2' of https://github.com/Agile86/ka…
revitalyr Jan 5, 2023
704befa
restore '.as_ref().unwrap()' for `RefCell<Option<`
revitalyr Jan 5, 2023
389ca15
refactoring.
Agile86 Jan 7, 2023
39b6635
translate_args: catch root and replace with *_rrc.
Agile86 Jan 19, 2023
628e3a5
unwrap Option.
Agile86 Jan 19, 2023
94dde6d
instanceCalculate use handleAssignmentSimple (fixed).
Agile86 Jan 24, 2023
8238825
catch opaque types (with Option wrapper).
Agile86 Jan 25, 2023
3784ad2
allow(overflowing_literals)
Agile86 Jan 26, 2023
e9d4555
store IO (as BytesReader) and use.
Agile86 Jan 30, 2023
2f03869
code cleanup.
Agile86 Jan 31, 2023
9359e84
support .as (doCast) for SwitchType enum (enum <-> variant).
Agile86 Feb 1, 2023
2e8fb7b
generate helper method with name from variant type only if there is o…
Agile86 Feb 1, 2023
ed2019e
doCast: check types without option always.
Agile86 Feb 1, 2023
16da2bf
ensure amp (&) for _io.
Agile86 Feb 1, 2023
aea84d3
pub _self introduced.
Agile86 Feb 2, 2023
e140e7e
correctly wrap parent type (nav_parent_override.ksy).
Agile86 Feb 2, 2023
58f792d
cast bugs fixed.
Agile86 Feb 4, 2023
4986dde
let rust decide final type for simple integers.
Agile86 Feb 4, 2023
a11a38b
Do logical right shift instead of arithmetic right shift.
Agile86 Feb 5, 2023
86071c8
debug info removed.
Agile86 Feb 5, 2023
c71120a
life params cleaned up.
Agile86 Feb 6, 2023
ed64b29
generate code that use OptRc.
Agile86 Feb 9, 2023
f15094c
code cleanup.
Agile86 Mar 6, 2023
0508222
unused code cleanup.
Agile86 Mar 10, 2023
27d84ea
no more rustc bug here. Gen better for reading code.
Agile86 Mar 10, 2023
31c904c
code cleanup.
Agile86 Mar 10, 2023
a043923
code cleanup.
Agile86 Mar 10, 2023
6eca299
dead code removed.
Agile86 Mar 10, 2023
8400eb8
useless comment removed.
Agile86 Mar 10, 2023
5e342f0
unused func removed.
Agile86 Mar 11, 2023
dca39ad
Merge remote-tracking branch 'origin/master' into rust_basic_support_v2
Agile86 Mar 11, 2023
75683ce
Revert "out full path of generated file"
Mar 11, 2023
619f380
reverted back to 1.7.1
Mar 11, 2023
8e230f3
Revert "add field `allClasses` to publish `classSpecs`"
Agile86 Mar 11, 2023
581dcf7
Revert "Report details if NotImplementedError."
Agile86 Mar 11, 2023
30f552f
code reverted.
Agile86 Mar 11, 2023
6b9aba6
added new field `allClasses` to publish classSpecs
Agile86 Mar 11, 2023
cd83719
space removed.
Agile86 Mar 13, 2023
b56c33d
Rust lang added.
Agile86 Mar 13, 2023
c85ec1e
sync changes with runtime/rust.
Agile86 Mar 26, 2023
2bef376
generate clean use-declaration.
Agile86 Dec 19, 2023
ab3f5ba
Merge branch 'master' of https://github.com/kaitai-io/kaitai_struct_c…
Agile86 Feb 16, 2024
9eb7efa
context-oriented method for searching members.
Agile86 Feb 16, 2024
091d38c
Merge branch 'master' into rust_basic_support_v2
generalmimon Apr 17, 2024
f1c697d
external type declaration fix.
Agile86 Jun 24, 2024
2332878
new feature interpolated strings implemented.
Agile86 Jun 24, 2024
24f04d9
Rust: fix ill-conceived insertion of `use` declarations
generalmimon May 9, 2024
f360132
Merge branch 'master' into rust_basic_support_v2
generalmimon Sep 1, 2024
d6793db
Rust: adapt to recent runtime library changes
generalmimon Sep 1, 2024
d7c9516
Rust: deny `arithmetic_overflow` and `overflowing_literals`
generalmimon Sep 2, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ resolvers ++= Resolver.sonatypeOssRepos("public")

val NAME = "kaitai-struct-compiler"
val VERSION = "0.11-SNAPSHOT"
val TARGET_LANGS = "C++/STL, C#, Go, Java, JavaScript, Lua, Nim, Perl, PHP, Python, Ruby"
val TARGET_LANGS = "C++/STL, C#, Go, Java, JavaScript, Lua, Nim, Perl, PHP, Python, Ruby, Rust"
val UTF8 = Charset.forName("UTF-8")

lazy val root = project.in(file(".")).
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import io.kaitai.struct.translators.TypeProvider

class ClassTypeProvider(classSpecs: ClassSpecs, var topClass: ClassSpec) extends TypeProvider {
var nowClass = topClass
val allClasses: ClassSpecs = classSpecs

var _currentIteratorType: Option[DataType] = None
var _currentSwitchType: Option[DataType] = None
Expand Down
48 changes: 34 additions & 14 deletions shared/src/main/scala/io/kaitai/struct/RustClassCompiler.scala
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
package io.kaitai.struct

import io.kaitai.struct.datatype.DataType.{KaitaiStreamType, UserTypeInstream}
import io.kaitai.struct.datatype.{Endianness, FixedEndian, InheritedEndian}
import io.kaitai.struct.datatype.DataType._
import io.kaitai.struct.datatype._
import io.kaitai.struct.exprlang.Ast
import io.kaitai.struct.format._
import io.kaitai.struct.languages.RustCompiler
import io.kaitai.struct.languages.components.ExtraAttrs
Expand Down Expand Up @@ -29,18 +30,19 @@ class RustClassCompiler(

// Basic struct declaration
lang.classHeader(curClass.name)

compileAttrDeclarations(curClass.seq ++ extraAttrs)
curClass.instances.foreach { case (instName, instSpec) =>
compileInstanceDeclaration(instName, instSpec)
}

// Constructor = Read() function
compileReadFunction(curClass)

compileInstances(curClass)

compileAttrReaders(curClass.seq ++ extraAttrs)
curClass.toStringExpr.foreach(expr => lang.classToString(expr))
lang.classFooter(curClass.name)

compileEnums(curClass)
Expand All @@ -49,7 +51,7 @@ class RustClassCompiler(
compileSubclasses(curClass)
}

def compileReadFunction(curClass: ClassSpec) = {
def compileReadFunction(curClass: ClassSpec): Unit = {
lang.classConstructorHeader(
curClass.name,
curClass.parentType,
Expand All @@ -58,23 +60,41 @@ class RustClassCompiler(
curClass.params
)

// FIXME
val defEndian = curClass.meta.endian match {
case Some(fe: FixedEndian) => Some(fe)
case _ => None
}

lang.readHeader(defEndian, false)


lang.readHeader(defEndian, isEmpty = false)

curClass.meta.endian match {
case Some(ce: CalcEndian) => compileCalcEndian(ce)
case Some(_) => // Nothing to generate
case None => // Same here
}

compileSeq(curClass.seq, defEndian)
lang.classConstructorFooter
}

override def compileInstances(curClass: ClassSpec) = {
override def compileCalcEndian(ce: CalcEndian): Unit = {
def renderProc(result: FixedEndian): Unit = {
val v = result match {
case LittleEndian => Ast.expr.IntNum(1)
case BigEndian => Ast.expr.IntNum(2)
}
lang.instanceCalculate(IS_LE_ID, CalcIntType, v)
}
lang.switchCases[FixedEndian](IS_LE_ID, ce.on, ce.cases, renderProc, renderProc)
lang.runReadCalc()
}

override def compileInstances(curClass: ClassSpec): Unit = {
lang.instanceDeclHeader(curClass.name)
curClass.instances.foreach { case (instName, instSpec) =>
compileInstance(curClass.name, instName, instSpec, curClass.meta.endian)
}
lang.instanceFooter
}

override def compileInstance(className: List[String], instName: InstanceIdentifier, instSpec: InstanceSpec, endian: Option[Endianness]): Unit = {
Expand All @@ -88,16 +108,16 @@ class RustClassCompiler(
lang.instanceHeader(className, instName, dataType, instSpec.isNullable)
lang.instanceCheckCacheAndReturn(instName, dataType)

lang.instanceSetCalculated(instName)
instSpec match {
case vi: ValueInstanceSpec =>
lang.attrParseIfHeader(instName, vi.ifExpr)
lang.instanceCalculate(instName, dataType, vi.value)
lang.attrParseIfFooter(vi.ifExpr)
case i: ParseInstanceSpec =>
lang.attrParse(i, instName, None) // FIXME
case pi: ParseInstanceSpec =>
lang.attrParse(pi, instName, None) // FIXME
}

lang.instanceSetCalculated(instName)
lang.instanceReturn(instName, dataType)
lang.instanceFooter
}
Expand Down
Loading
Loading