Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: remove UI #2036

Merged
merged 6 commits into from
Dec 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .github/workflows/markdown-lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@ jobs:
with:
globs: |
**/*.md
!ui/node_modules
!LICENSE.md
!pkg/web/openapi/**
!.github/*.md
Expand Down
35 changes: 0 additions & 35 deletions .github/workflows/ui-lint.yml

This file was deleted.

35 changes: 0 additions & 35 deletions .github/workflows/ui-tests.yml

This file was deleted.

35 changes: 0 additions & 35 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -28,41 +28,6 @@ conduit.db
/conduit
/conduit-pipeline-check

### Conduit UI ###

# compiled output
/ui/dist/
/ui/tmp/
/pkg/web/ui/dist/*
!/pkg/web/ui/dist/.keep

# dependencies
/.github/actions/node_modules/
/ui/node_modules/

# misc
/ui/.env*
/ui/.pnp*
/ui/.sass-cache
/ui/.eslintcache
/ui/connect.lock
/ui/coverage/
/ui/libpeerconnection.log
/ui/npm-debug.log*
/ui/testem.log
/ui/yarn-error.log

# ember-try
/ui/.node_modules.ember-try/
/ui/bower.json.ember-try
/ui/npm-shrinkwrap.json.ember-try
/ui/package.json.ember-try
/ui/package-lock.json.ember-try
/ui/yarn.lock.ember-try

# Local Netlify folder
/ui/.netlify

### OS ###
.DS_Store

Expand Down
1 change: 0 additions & 1 deletion .golangci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,6 @@ issues:
exclude-dirs-use-default: false
exclude-dirs:
- ^examples/
- ^ui/
- ^pkg/plugin/processor/builtin/internal/diff # external code

exclude-rules:
Expand Down
6 changes: 0 additions & 6 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,6 @@ RUN apt-get update &&\
apt-get install -y build-essential &&\
apt-get install -y git

# Install Node@v18
RUN curl -sL https://deb.nodesource.com/setup_18.x | bash - &&\
apt-get install -y nodejs &&\
npm update &&\
npm i -g [email protected]

# Build the full app binary
WORKDIR /app
COPY . .
Expand Down
18 changes: 3 additions & 15 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ GO_VERSION_CHECK=`./scripts/check-go-version.sh`

# The build target should stay at the top since we want it to be the default target.
.PHONY: build
build: check-go-version pkg/web/ui/dist
go build -ldflags "-X 'github.com/conduitio/conduit/pkg/conduit.version=${VERSION}'" -o conduit -tags ui ./cmd/conduit/main.go
build: check-go-version
go build -ldflags "-X 'github.com/conduitio/conduit/pkg/conduit.version=${VERSION}'" -o conduit ./cmd/conduit/main.go
@echo "\nBuild complete. Enjoy using Conduit!"
@echo "Get started by running:"
@echo " ./conduit"
Expand Down Expand Up @@ -40,11 +40,6 @@ fmt:
lint:
golangci-lint run -v

.PHONY: build-server
build-server: check-go-version
go build -ldflags "-X 'github.com/conduitio/conduit/pkg/conduit.version=${VERSION}'" -o conduit ./cmd/conduit/main.go
@echo "build version: ${VERSION}"

.PHONY: run
run:
go run ./cmd/conduit/main.go
Expand All @@ -64,7 +59,6 @@ proto-lint:
.PHONY: clean
clean:
@rm -f conduit
@rm -rf pkg/web/ui/dist

.PHONY: download
download:
Expand All @@ -81,12 +75,6 @@ install-tools: download
generate:
go generate -x ./...

pkg/web/ui/dist:
make ui-dist

ui-%:
@cd ui && make $*

.PHONY: check-go-version
check-go-version:
@if [ "${GO_VERSION_CHECK}" != "" ]; then\
Expand All @@ -96,4 +84,4 @@ check-go-version:

.PHONY: markdown-lint
markdown-lint:
markdownlint-cli2 "**/*.md" "#ui/node_modules" "#LICENSE.md" "#pkg/web/openapi/**" "#.github/*.md"
markdownlint-cli2 "**/*.md" "#LICENSE.md" "#pkg/web/openapi/**" "#.github/*.md"
66 changes: 8 additions & 58 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,7 @@ _Data Integration for Production Data Stores. :dizzy:_

Conduit is a data streaming tool written in Go. It aims to provide the best user
experience for building and running real-time data pipelines. Conduit comes with
batteries included, it provides a UI, common connectors, processors and
observability data out of the box.
common connectors, processors and observability data out of the box.

Conduit pipelines are built out of simple building blocks which run in their own
goroutines and are connected using Go channels. This makes Conduit pipelines
Expand All @@ -40,43 +39,12 @@ Conduit was created and open-sourced by [Meroxa](https://meroxa.io).
- [Connectors](#connectors)
- [Processors](#processors)
- [API](#api)
- [UI](#ui)
- [Documentation](#documentation)
- [Contributing](#contributing)

## Quick start

1. Download and extract
the [latest release](https://github.com/conduitio/conduit/releases/latest).
2. Download
the [example pipeline](/examples/pipelines/file-to-file.yaml)
and put it in the directory named `pipelines` in the same directory as the
Conduit binary.
3. Run Conduit (`./conduit`). The example pipeline will start automatically.
4. Write something to file `example.in` in the same directory as the Conduit
binary.

```sh
echo "hello conduit" >> example.in
```

5. Read the contents of `example.out` and notice an OpenCDC record:

```sh
$ cat example.out
{"position":"MTQ=","operation":"create","metadata":{"file.path":"./example.in","opencdc.readAt":"1663858188836816000","opencdc.version":"v1"},"key":"MQ==","payload":{"before":null,"after":"aGVsbG8gY29uZHVpdA=="}}
```

6. The string `hello conduit` is a base64 encoded string stored in the field
`payload.after`, let's decode it:

```sh
$ cat example.out | jq ".payload.after | @base64d"
"hello conduit"
```

7. Explore the UI by opening `http://localhost:8080` and build your own
pipeline!
<https://conduit.io/docs/getting-started>

## Installation guide

Expand All @@ -90,8 +58,8 @@ simply run it!
./conduit
```

Once you see that the service is running you may access a user-friendly web
interface at `http://localhost:8080`. You can also interact with
Once you see that the service is running, the configured pipeline should start
processing records automatically. You can also interact with
the [Conduit API](#api) directly, we recommend navigating
to `http://localhost:8080/openapi` and exploring the HTTP API through Swagger
UI.
Expand Down Expand Up @@ -132,9 +100,6 @@ rpm -i conduit_0.12.2_Linux_x86_64.rpm
Requirements:

- [Go](https://golang.org/)
- [Node.js](https://nodejs.org/) (18.x)
- [Yarn](https://yarnpkg.com/) (latest 1.x)
- [Ember CLI](https://ember-cli.com/)
- [Make](https://www.gnu.org/software/make/)

```shell
Expand All @@ -144,11 +109,6 @@ make
./conduit
```

Note that you can also build Conduit with `make build-server`, which only
compiles the server and skips the UI. This command requires only Go and builds
the binary much faster. That makes it useful for development purposes or for
running Conduit as a simple backend service.

### Docker

Our Docker images are hosted on GitHub's Container Registry. To run the latest
Expand All @@ -158,9 +118,6 @@ Conduit version, you should run the following command:
docker run -p 8080:8080 conduit.docker.scarf.sh/conduitio/conduit:latest
```

The Docker image includes the [UI](#ui), you can access it by navigating
to `http://localhost:8080`.

## Configuring Conduit

Conduit accepts CLI flags, environment variables and a configuration file to
Expand Down Expand Up @@ -190,6 +147,10 @@ each configuration option based on the following priorities:
connection-string: postgres://localhost:5432/conduitdb # -db.postgres.connection-string or CONDUIT_DB_POSTGRES_CONNECTION_STRING
```

This parsing configuration is provided thanks to our own CLI library [ecdysis](https://github.com/conduitio/ecdysis),
which builds on top of [Cobra](https://github.com/spf13/cobra) and uses [Viper](https://github.com/spf13/viper)
under the hood.

## Storage

Conduit's own data (information about pipelines, connectors, etc.) can be stored
Expand Down Expand Up @@ -296,17 +257,6 @@ or run Conduit and navigate to `http://localhost:8080/openapi` to open
a [Swagger UI](https://github.com/swagger-api/swagger-ui) which makes it easy to
try it out.

## UI

Conduit comes with a web UI that makes building data pipelines a breeze, you can
access it at `http://localhost:8080`. See
the [installation guide](#build-from-source) for instructions on how to build
Conduit with the UI.

For more information about the UI refer to the [Readme](ui/README.md) in `/ui`.

![animation](docs/data/animation.gif)

## Documentation

To learn more about how to use Conduit
Expand Down
Binary file removed docs/data/animation.gif
Binary file not shown.
1 change: 0 additions & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ require (
buf.build/gen/go/grpc-ecosystem/grpc-gateway/protocolbuffers/go v1.36.0-20240617172850-a48fcebcf8f1.1
github.com/Masterminds/semver/v3 v3.3.1
github.com/Masterminds/sprig/v3 v3.3.0
github.com/NYTimes/gziphandler v1.1.1
github.com/bufbuild/buf v1.47.2
github.com/conduitio/conduit-commons v0.5.0
github.com/conduitio/conduit-connector-file v0.9.0
Expand Down
2 changes: 0 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -94,8 +94,6 @@ github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERo
github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU=
github.com/Microsoft/hcsshim v0.12.9 h1:2zJy5KA+l0loz1HzEGqyNnjd3fyZA31ZBCGKacp6lLg=
github.com/Microsoft/hcsshim v0.12.9/go.mod h1:fJ0gkFAna6ukt0bLdKB8djt4XIJhF/vEPuoIWYVvZ8Y=
github.com/NYTimes/gziphandler v1.1.1 h1:ZUDjpQae29j0ryrS0u/B8HZfJBtBQHjqw2rQ2cqUQ3I=
github.com/NYTimes/gziphandler v1.1.1/go.mod h1:n/CVRwUEOgIxrgPvAQhUUr9oeUtvrhMomdKFjzJNB0c=
github.com/OpenPeeDeeP/depguard/v2 v2.2.0 h1:vDfG60vDtIuf0MEOhmLlLLSzqaRM8EMcgJPdp74zmpA=
github.com/OpenPeeDeeP/depguard/v2 v2.2.0/go.mod h1:CIzddKRvLBC4Au5aYP/i3nyaWQ+ClszLIuVocRiCYFQ=
github.com/alecthomas/assert/v2 v2.2.2 h1:Z/iVC0xZfWTaFNE6bA3z07T86hd45Xe2eLt6WVy2bbk=
Expand Down
31 changes: 0 additions & 31 deletions pkg/conduit/runtime.go
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,6 @@ import (
"github.com/conduitio/conduit/pkg/schemaregistry"
"github.com/conduitio/conduit/pkg/web/api"
"github.com/conduitio/conduit/pkg/web/openapi"
"github.com/conduitio/conduit/pkg/web/ui"
apiv1 "github.com/conduitio/conduit/proto/api/v1"
grpcruntime "github.com/grpc-ecosystem/grpc-gateway/v2/runtime"
"github.com/piotrkowalczuk/promgrpc/v4"
Expand Down Expand Up @@ -352,7 +351,6 @@ func (r *Runtime) Run(ctx context.Context) (err error) {
port = tcpAddr.Port
}
r.logger.Info(ctx).Send()
r.logger.Info(ctx).Msgf("click here to navigate to Conduit UI: http://localhost:%d/ui", port)
r.logger.Info(ctx).Msgf("click here to navigate to explore the HTTP API: http://localhost:%d/openapi", port)
r.logger.Info(ctx).Send()
}
Expand Down Expand Up @@ -676,35 +674,6 @@ func (r *Runtime) serveHTTPAPI(
return nil, cerrors.Errorf("failed to register openapi redirect handler: %w", err)
}

uiHandler, err := ui.Handler()
if err != nil {
return nil, cerrors.Errorf("failed to set up ui handler: %w", err)
}

uiHandler = http.StripPrefix("/ui", uiHandler)

err = gwmux.HandlePath(
"GET",
"/ui/**",
func(w http.ResponseWriter, req *http.Request, pathParams map[string]string) {
uiHandler.ServeHTTP(w, req)
},
)
if err != nil {
return nil, cerrors.Errorf("failed to register ui handler: %w", err)
}

err = gwmux.HandlePath(
"GET",
"/",
func(w http.ResponseWriter, req *http.Request, pathParams map[string]string) {
http.Redirect(w, req, "/ui", http.StatusFound)
},
)
if err != nil {
return nil, cerrors.Errorf("failed to register redirect handler: %w", err)
}

metricsHandler := r.newHTTPMetricsHandler()
err = gwmux.HandlePath(
"GET",
Expand Down
Loading
Loading