Skip to content

Commit

Permalink
Lots of uml2ts stuff (#33)
Browse files Browse the repository at this point in the history
* Guess what? More restructuring!

* Update workflow paths

* Just stuff and things

* It kinda does the thing in a way

* Buf stuff

* Regen stuff

* Little tidbits in the TS

* Fix lockfile again

* Taking the stuff from the other thing

* Working on the thing or something. Idfk

* Make it all pretty and compilable

* Refactor generator test for simplicity

The generator test has been simplified by removing the use of a Writable stream. The 'gen' function now directly returns the result, which is then tested for null and emptiness. This change makes the test more straightforward and easier to understand.

* Added test command to Makefile

A new 'test' command has been added to the Makefile. This allows for running tests directly from the make utility, improving the testing workflow.

* Added nested fields to 'test' type

The 'test' type in the Generator now supports nested fields. A new field named 'test' of type string has been added under the existing 'test' type. This enhancement provides more flexibility and depth when defining types.

* Refactored Makefile for better organization

The Makefile has been restructured to improve readability and maintainability. Changes include:
- Separated build and test targets into dotnet-specific and package-specific ones
- Reordered clean targets for logical consistency
- Moved proto-related operations into their own targets with dependencies
- Added explicit touch commands to ensure correct timestamp updates on build artifacts

* Expanded MIME type support in UML generator

The UML generator's read function now supports a wider range of MIME types. A new 'SupportedMimeType' type has been introduced to handle this change. The function will now throw an error for unrecognized media types, improving the robustness of the code.

* Updated package.json in UML package

The package.json file in the UML package has been updated. The 'module' field was moved under the 'type' field for better organization. Additionally, two new fields were added: 'main', which points to the main JavaScript file, and 'types', which points to the TypeScript definition file.

* Added fast-check to devDependencies

The package.json file in the UML package has been updated. The main change is the addition of "fast-check" to the list of development dependencies, which will help with property-based testing.

* Added test step to build action

A new testing step has been added to the GitHub Actions workflow for building. This step runs after the build process and before linting, ensuring that tests are executed as part of the continuous integration pipeline.

* Added tests and updated MIME types handling

- Introduced new tests for the 'read' function in the UML package, checking its ability to handle different data types.
- Refactored SupportedMimeType from a union type to an array constant and derived tuple type. This change enhances code maintainability by centralizing MIME type definitions.

* Refactored UML to TypeScript conversion logic

The code has been refactored to improve the structure and readability. The main changes include:
- Extracted the generation logic into a separate command file.
- Introduced an Io interface for better handling of input/output operations.
- Simplified index.ts by using a new app function that applies commands.
- Created a new program.ts file to handle command-related operations, including options and actions.

* Updated dependencies in package.json

Added 'fast-check' to the devDependencies in the package.json file. This new dependency will help improve testing capabilities. Also, there's a binary change in bun.lockb file.

* Updated MIME type in test cases

The MIME type 'application/protobuf' was replaced with 'application/x-protobuf' in the unit tests. This change ensures that our tests are aligned with the correct and updated MIME types.

* Refactor IO interface and streamline gen function

Significant changes include:
- Replaced 'BunFile' with 'ArrayBufferSink' and 'Blob' in the Io interface
- Removed check for stdin readability in the gen function
- Simplified writing to stdout in the gen function

* Added support for protobuf and JSON data

Implemented functionality to read protobuf and JSON data in the 'gen' function. This includes creating arbitrary fields and types, converting them into binary or JSON format, and then reading them back. Also added tests to verify this functionality for different mime types of protobuf as well as JSON data.

* Updated dependencies in package.json

Added 'testcontainers' to the devDependencies in the package.json file. This new dependency will aid in managing test environments.

* Add Docker support for uml2ts

Introduced Docker support for the uml2ts project. This includes:
- A new Makefile to build the Docker image
- A .dockerignore file to exclude unnecessary files from the Docker context
- A multi-stage Dockerfile that sets up the necessary environment, copies over required files, installs dependencies, and builds the application.

* Updated package.json and bun.lockb

Removed 'src/*' from the workspaces array in package.json. Also, updated the binary file bun.lockb.

* Refactor IO handling in uml2ts package

Significant changes include:
- Removed the Io interface and its usage from command.ts and program.ts
- Directly used Bun's stdin and stdout for buffer reading and writing in gen function
- Simplified action call in gen command by removing io() argument

* Added Makefile and updated package.json

A new Makefile has been added to the uml2ts package, introducing build, test, clean, and docker commands. The package.json file in the same package has also been updated with a removal of the "prestart" script.

* Updated Docker build process

The Docker build process has been updated to include a new test target. This change introduces a new 'uml2ts_test' target in the Makefile, which builds a Docker image for testing purposes. Additionally, the Dockerfile now includes an extra stage that uses Ubuntu as its base image for this test build.

* Added test_image target to Makefile

A new target, 'test_image', has been added to the Makefile. This allows for the creation of a test image using the existing docker setup.

* Updated Dockerfile for uml2ts

The Dockerfile for uml2ts has been updated to include a build platform argument. The base image for the test stage has also been changed to a specific version of Ubuntu, and the final stage now includes an alias.

* Updated test setup and command execution

The testing setup has been updated to include a `beforeAll` block that builds the necessary binary file for tests. The way commands are executed in tests has also been changed, replacing the previous IO-based approach with spawning processes. This change simplifies the code and makes it easier to manage input/output streams.

In addition, there's a minor refactor in how commands are applied in the main application logic, making it more functional by passing root as a function instead of an object.

* Refactor test setup and simplify tests

The test setup has been refactored to ensure a clean environment before each run. The 'Field' and 'Type' imports have been removed, as well as the corresponding arbitrary generators. Tests have been simplified by removing the use of fast-check library and instead using static values for testing. An 'afterAll' cleanup function has also been added to remove any residual files after tests are completed.

* Fixes

* More fixes

* Format

* And some more
  • Loading branch information
UnstoppableMango authored May 2, 2024
1 parent 22ae98a commit 7ccae17
Show file tree
Hide file tree
Showing 41 changed files with 3,992 additions and 253 deletions.
5 changes: 5 additions & 0 deletions .github/actions/build-bun/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,11 @@ runs:
working-directory: ${{ inputs.path }}
run: bun run build

- name: Test
shell: bash
working-directory: ${{ inputs.path }}
run: bun test

- name: Lint
shell: bash
# This will likely error on unrelated linting errors...
Expand Down
8 changes: 7 additions & 1 deletion .github/workflows/packages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,12 +29,18 @@ jobs:

build-bun:
runs-on: ubuntu-latest
strategy:
matrix:
path:
- ts
- uml
- uml2ts
steps:
- uses: actions/checkout@v4

- uses: ./.github/actions/build-bun
with:
path: packages/ts
path: packages/${{ matrix.path }}

build-go:
runs-on: ubuntu-latest
Expand Down
59 changes: 35 additions & 24 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -23,28 +23,33 @@ CLI_BIN := $(CLI_DIR)/$(BIN_PATH)/um.dll
LANG_DIR := src/Language
LANG_SRC := $(shell find $(LANG_DIR) -name '*.fs' -not -path '*obj*' -type f)

.PHONY: build test gen lint
build: $(LANG_SRC) $(CLI_SRC) $(BROKER_SRC) build_proto
.PHONY: build build_dotnet
build: build_dotnet
@touch .make/build_lang
dotnet build
build_dotnet: .make/build_dotnet

test: build
.PHONY: test test_dotnet
test: test_dotnet test_packages
test_dotnet: build_dotnet
dotnet test --no-build
test_packages:
@$(MAKE) -C packages test

gen: clean_gen build_proto
buf generate
.PHONY: gen
gen: gen_proto

.PHONY: lint
lint: .make/lint_proto .make/lint_lang

.PHONY: clean clean_gen clean_src clean_dist
clean: clean_gen clean_src clean_dist
rm -rf .make
clean_src:
@$(MAKE) -C src clean
clean_cli:
@$(MAKE) -C cli clean
clean_gen:
@$(MAKE) -C gen clean
clean_src:
@$(MAKE) -C src clean
clean_dist:
@find . -type d -name dist \
-not -path '*node_modules*' \
Expand All @@ -61,12 +66,10 @@ tidy: gen
release:
goreleaser release --snapshot --clean

.PHONY: build_proto
build_proto:
buf build

$(BROKER_BIN): $(BROKER_SRC)
dotnet build ${BROKER_DIR}
.PHONY: proto gen_proto build_proto
proto: build_proto gen_proto
gen_proto: .make/gen_proto
build_proto: .make/build_proto

# The naming is kinda silly but its short
.PHONY: work
Expand All @@ -87,22 +90,30 @@ go.work.sum: go.work
dotnet tool restore
@touch $@

PROTO_SRC := $(shell find proto -type f -name '*.proto')
.make/gen_proto: .make/build_proto $(PROTO_SRC)
buf generate
@touch $@
.make/build_proto: $(PROTO_SRC)
buf build
@touch $@
.make/lint_proto: $(PROTO_SRC)
buf lint proto
@touch $@

.make/build_dotnet: $(LANG_SRC) $(BROKER_SRC) $(CLI_SRC) .make/gen_proto
dotnet build
@touch $@ .make/build_lang .make/build_broker .make/build_cli
.make/build_lang: $(LANG_SRC)
dotnet build ${LANG_DIR}
@touch $@

.make/build_cli: $(CLI_SRC)
dotnet build ${CLI_DIR}
.make/build_broker: $(BROKER_SRC) .make/gen_proto
dotnet build ${BROKER_DIR}
@touch $@

.make/lint_proto:
buf lint proto
.make/build_cli: $(CLI_SRC) .make/gen_proto
dotnet build ${CLI_DIR}
@touch $@

.make/lint_lang: .make/tool_restore $(LANG_SRC)
dotnet fantomas ${LANG_DIR}
@touch $@

.make/build_plugin_gen_ts:
cd plugin/gen/ts && bun run build
@touch $@
2 changes: 2 additions & 0 deletions buf.gen.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ managed:
enabled: true
go_package_prefix:
default: github.com/unstoppablemango/tdl/gen/proto/go
except:
- buf.build/googleapis/googleapis

plugins:
- plugin: buf.build/connectrpc/go:v1.16.1
Expand Down
Binary file modified bun.lockb
Binary file not shown.
15 changes: 15 additions & 0 deletions docker/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
CONTEXT := $(shell git rev-parse --show-toplevel)

.PHONY: uml2ts uml2ts_test
uml2ts:
docker build \
-f uml2ts/Dockerfile \
-t unstoppablemango/uml2ts \
${CONTEXT}

uml2ts_test:
docker build \
-f uml2ts/Dockerfile \
-t unstoppablemango/uml2ts_test \
--target test \
${CONTEXT}
22 changes: 22 additions & 0 deletions docker/uml2ts/.dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
node_modules
Dockerfile*
docker-compose*
.dockerignore
.git
.gitignore
README.md
LICENSE
.vscode
Makefile
.env
.editorconfig
.idea
coverage*
.config
.github
.make
cli
docker
pkg
proto
src
35 changes: 35 additions & 0 deletions docker/uml2ts/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
FROM oven/bun:1 AS base
ARG BUILDPLATFORM
WORKDIR /build

FROM base AS install

RUN mkdir -p gen/proto/{es,ts} packages/{ts,uml,uml2ts}

COPY package.json .
COPY gen/proto/es/package.json gen/proto/es/
COPY gen/proto/ts/package.json gen/proto/ts/
COPY packages/ts/package.json packages/ts/
COPY packages/uml/package.json packages/uml/
COPY packages/uml2ts/package.json packages/uml2ts/
COPY bun.lockb .

RUN bun install --frozen-lockfile --production

FROM --platform=$BUILDPLATFORM install AS build
COPY --from=install /build/node_modules .

COPY gen/proto/es/ gen/proto/es/
COPY gen/proto/ts/ gen/proto/ts/
COPY packages/ts/ packages/ts/
COPY packages/uml/ packages/uml/
COPY packages/uml2ts/ .

RUN bun run build

FROM ubuntu:noble-20240429 AS test
COPY --from=build /build/dist/uml2ts .

FROM --platform=$BUILDPLATFORM scratch AS dist
COPY --from=build /build/dist/uml2ts .
ENTRYPOINT ["uml2ts"]
7 changes: 4 additions & 3 deletions eslint.config.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,11 @@ export default tslint.config(
'.idea/',
'.make',
'.vscode/',
'bin/',
'gen/',
'obj/',
'proto/',
'**/bin/',
'**/gen/',
'**/obj/',
'**/dist/',
],
},
{
Expand Down
Loading

0 comments on commit 7ccae17

Please sign in to comment.