Skip to content

Commit

Permalink
1.0.0 multirow inserts, pointers, streamlined usage (#7)
Browse files Browse the repository at this point in the history 
* Use pkg directory for source code following convention

* Rename files for Insert to insert.go in preparation for MultiInsert

* Support for multi row insert

* Flesh out README

* Refinements to README

* README example is a function not a routine

* Refactor multirow insert handler for clarity. Put Insert-related interfaces in the same module.

* Multirow insert is stable. Unit tests for all variations: single and multi row; struct, struct-pointer, slice-of-struct, slice-of-struct-pointer.

* Refine README headers - what this is not, what this is

Co-authored-by: Zach Victor <[email protected]>
  • Loading branch information
@zachvictor
zachvictor and Zach Victor authored Aug 6, 2022
1 parent 4341e7c commit f27b2f9
Show file tree
Hide file tree
Showing 8 changed files with 793 additions and 434 deletions.
99 changes: 78 additions & 21 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,16 +1,18 @@
# sqlinsert
Generate SQL INSERT statement with bind parameters directly from a Go struct.
Generate a SQL INSERT statement with bind parameters directly from a Go struct.
[![Go Reference](https://pkg.go.dev/badge/github.com/zachvictor/sqlinsert.svg)](https://pkg.go.dev/github.com/zachvictor/sqlinsert)

## Features
* Define column names in struct tags.
* Struct values become bind arguments.
* Use SQL outputs and Args slice piecemeal. Or, use `Insert()`/`InsertContext()` with a `sql.Conn`, `sql.DB`, or `sql.Tx` to execute the INSERT statement directly, all in one blow.
* Use SQL outputs and Args slice piecemeal. Or, use `Insert()`/`InsertContext()` with a `sql.Conn`, `sql.DB`, or
`sql.Tx` to execute the INSERT statement directly.
* Works seamlessly with Go standard library [database/sql](https://pkg.go.dev/database/sql) package.
* Supports bind parameter token types of MySQL, PostgreSQL, Oracle, SingleStore (MemSQL), SQL Server (T-SQL), and their equivalents.
* Supports bind parameter token types of MySQL, PostgreSQL, Oracle, SingleStore (MemSQL), SQL Server (T-SQL), and their
equivalents.
* Supports customized struct tags and token types.
* Supports Go 1.15 and later.
* 100% unit test coverage.
* Supports Go 1.8 to 1.19.
* Test coverage: 100% files, 97.5% statements. Tested on Go 1.15, 1.17, and 1.18.

## Example
### Given
Expand All @@ -27,17 +29,17 @@ CREATE TABLE candy (
```

```go
type candyInsert struct {
Id string `col:"id"`
Name string `col:"candy_name"`
FormFactor string `col:"form_factor"`
Description string `col:"description"`
Mfr string `col:"manufacturer"`
Weight float64 `col:"weight_grams"`
Timestamp time.Time `col:"ts"`
type CandyInsert struct {
Id string `col:"id"`
Name string `col:"candy_name"`
FormFactor string `col:"form_factor"`
Description string `col:"description"`
Mfr string `col:"manufacturer"`
Weight float64 `col:"weight_grams"`
Timestamp time.Time `col:"ts"`
}

var rec = candyInsert{
var rec = CandyInsert{
Id: `c0600afd-78a7-4a1a-87c5-1bc48cafd14e`,
Name: `Gougat`,
FormFactor: `Package`,
Expand All @@ -50,16 +52,71 @@ var rec = candyInsert{

### Before
```go
stmt, _ := db.Prepare(`INSERT INTO candy
(id, candy_name, form_factor, description, manufacturer, weight_grams, ts)
VALUES ($1, $2, $3, $4, $5, $6, $7)`)
_, err := stmt.Exec(candyInsert.Id, candyInsert.Name, candyInsert.FormFactor,
stmt, _ := db.Prepare(`INSERT INTO candy
(id, candy_name, form_factor, description, manufacturer, weight_grams, ts)
VALUES (?, ?, ?, ?, ?, ?, ?)`)
_, err := stmt.Exec(candyInsert.Id, candyInsert.Name, candyInsert.FormFactor,
candyInsert.Description, candyInsert.Mfr, candyInsert.Weight, candyInsert.Timestamp)
```

### After
```go
sqlinsert.UseTokenType = OrdinalNumberTokenType
ins := sqlinsert.NewInsert(`candy`, rec)
ins := sqlinsert.Insert{`candy`, &rec}
_, err := ins.Insert(db)
```
```

## This is not an ORM

### Hide nothing
Unlike ORMs, `sqlinsert` does **not** create an abstraction layer over SQL relations, nor does it restructure SQL
functions.
The aim is to keep it simple and hide nothing.
`sqlinsert` is fundamentally a helper for [database/sql](https://pkg.go.dev/database/sql).
It simply maps struct fields to INSERT elements:
* struct tags
=> SQL columns and tokens `string`
=> [Prepare](https://pkg.go.dev/database/[email protected]#DB.Prepare) `query string`
* struct values
=> bind args `[]interface{}`
=> [Exec](https://pkg.go.dev/database/[email protected]#Stmt.Exec) `args ...interface{}`
([Go 1.18](https://pkg.go.dev/database/[email protected]#DB.ExecContext)+ `args ...any`)

### Use only what you need
All aspects of SQL INSERT remain in your control:
* *I just want the column names for my SQL.* `Insert.Columns()`
* *I just want the parameter-tokens for my SQL.* `Insert.Params()`
* *I just want the bind args for my Exec() call.* `Insert.Args()`
* *I just want a Prepare-Exec wrapper.* `Insert.Insert()`

## This is a helper

### Let SQL be great
SQL’s INSERT is already as close to functionally pure as possible. Why would we change that? Its simplicity and
directness are its power.

### Let database/sql be great
Some database vendors support collection types for bind parameters, some don’t.
Some database drivers support slices for bind args, some don’t.
The complexity of this reality is met admirably by [database/sql](https://pkg.go.dev/database/sql)
with the _necessary_ amount of flexibility and abstraction:
*flexibility* in open-ended SQL;
*abstraction* in the variadic `args ...interface{}` for bind args.
In this way, [database/sql](https://pkg.go.dev/database/sql) respects INSERT’s power,
hiding nothing even as it tolerates the vagaries of bind-parameter handling among database vendors and drivers.

### Let Go be great
Go structs support ordered fields, strong types, and field metadata via [tags](https://go.dev/ref/spec#Tag) and
[reflection](https://pkg.go.dev/reflect#StructTag).
In these respects, the Go struct can encapsulate the information of a SQL INSERT-row perfectly and completely.
`sqlinsert` uses these features of Go structs to makes your SQL INSERT experience more Go-idiomatic.

## Limitations of the Prepare-Exec wrappers
`Insert.Insert` and `Insert.Context` are for simple binding *only.*
In the spirit of "hide nothing," these do *not* support SQL operations in the `VALUES` clause.
If you require, say—
```sql
INSERT INTO foo (bar, baz, oof) VALUES (some_function(?), REPLACE(?, 'oink', 'moo'), ? + ?);
```
—then you can use `sqlinsert.Insert` methods piecemeal.
For example, use `Insert.Columns` to build the column list for `Prepare`
and `Insert.Args` to marshal the args for `Exec`/`ExecContext`.
165 changes: 165 additions & 0 deletions pkg/insert.go
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,165 @@
package sqlinsert

import (
"context"
"database/sql"
"fmt"
"reflect"
"strings"
)

// InsertWith models functionality needed to execute a SQL INSERT statement with database/sql via sql.DB or sql.Tx.
// Note: sql.Conn is also supported, however, for PrepareContext and ExecContext only.
type InsertWith interface {
Prepare(query string) (*sql.Stmt, error)
PrepareContext(ctx context.Context, query string) (*sql.Stmt, error)
Exec(query string, args ...interface{}) (sql.Result, error)
ExecContext(ctx context.Context, query string, args ...interface{}) (sql.Result, error)
}

// Inserter models functionality to produce a valid SQL INSERT statement with bind args.
type Inserter interface {
Tokenize(tokenType TokenType) string
Columns() string
Params() string
SQL() string
Args() []interface{}
Insert(with InsertWith) (*sql.Stmt, error)
InsertContext(ctx context.Context, with InsertWith) (*sql.Stmt, error)
}

// Insert models data used to produce a valid SQL INSERT statement with bind args.
// Table is the table name. Data is either a struct with column-name tagged fields and the data to be inserted or
// a slice struct (struct ptr works too). Private recordType and recordValue fields are used with reflection to get
// struct tags for Insert.Columns, Insert.Params, and Insert.SQL and to retrieve values for Insert.Args.
type Insert struct {
Table string
Data interface{}
}

// Columns returns the comma-separated list of column names-as-tokens for the SQL INSERT statement.
// Multi Row Insert: Insert.Data is a slice; first item in slice is
func (ins *Insert) Columns() string {
v := reflect.ValueOf(ins.Data)
if v.Kind() == reflect.Slice {
if v.Index(0).Kind() == reflect.Pointer {
return Tokenize(v.Index(0).Elem().Type(), ColumnNameTokenType)
} else {
return Tokenize(v.Index(0).Type(), ColumnNameTokenType)
}
} else if v.Kind() == reflect.Pointer {
return Tokenize(v.Elem().Type(), ColumnNameTokenType)
} else {
return Tokenize(v.Type(), ColumnNameTokenType)
}
}

// Params returns the comma-separated list of bind param tokens for the SQL INSERT statement.
func (ins *Insert) Params() string {
v := reflect.ValueOf(ins.Data)
if v.Kind() == reflect.Slice {
var (
b strings.Builder
paramRow string
)
if v.Index(0).Kind() == reflect.Pointer {
paramRow = Tokenize(v.Index(0).Elem().Type(), UseTokenType)
} else {
paramRow = Tokenize(v.Index(0).Type(), UseTokenType)
}
b.WriteString(paramRow)
for i := 1; i < v.Len(); i++ {
b.WriteString(`,`)
b.WriteString(paramRow)
}
return b.String()
} else if v.Kind() == reflect.Pointer {
return Tokenize(v.Elem().Type(), UseTokenType)
} else {
return Tokenize(v.Type(), UseTokenType)
}
}

// SQL returns the full parameterized SQL INSERT statement.
func (ins *Insert) SQL() string {
var insertSQL strings.Builder
_, _ = fmt.Fprintf(&insertSQL, `INSERT INTO %s %s VALUES %s`,
ins.Table, ins.Columns(), ins.Params())
return insertSQL.String()
}

// Args returns the arguments to be bound in Insert() or the variadic Exec/ExecContext functions in database/sql.
func (ins *Insert) Args() []interface{} {
var (
data reflect.Value
rec reflect.Value
recType reflect.Type
args []interface{}
)
data = reflect.ValueOf(ins.Data)
if data.Kind() == reflect.Slice { // Multi row INSERT: Insert.Data is a slice-of-struct-pointer or slice-of-struct
argIndex := -1
if data.Index(0).Kind() == reflect.Pointer { // First slice element is struct pointers
recType = data.Index(0).Elem().Type()
} else { // First slice element is struct
recType = data.Index(0).Type()
}
numRecs := data.Len()
numFieldsPerRec := recType.NumField()
numBindArgs := numRecs * numFieldsPerRec
args = make([]interface{}, numBindArgs)
for rowIndex := 0; rowIndex < data.Len(); rowIndex++ {
if data.Index(0).Kind() == reflect.Pointer {
rec = data.Index(rowIndex).Elem() // Cur slice elem is struct pointer, get arg val from ref-element
} else {
rec = data.Index(rowIndex) // Cur slice elem is struct, can get arg val directly
}
for fieldIndex := 0; fieldIndex < numFieldsPerRec; fieldIndex++ {
argIndex += 1
args[argIndex] = rec.Field(fieldIndex).Interface()
}
}
return args
} else { // Single-row INSERT: Insert.Data must be a struct pointer or struct (otherwise reflect will panic)
if data.Kind() == reflect.Pointer { // Row information via struct pointer
recType = data.Elem().Type()
rec = data.Elem()
} else { // Row information via struct
recType = data.Type()
rec = data
}
args = make([]interface{}, recType.NumField())
for i := 0; i < recType.NumField(); i++ {
args[i] = rec.Field(i).Interface()
}
return args
}
}

// Insert prepares and executes a SQL INSERT statement on a *sql.DB, *sql.Tx,
// or other Inserter-compatible interface to Prepare and Exec.
func (ins *Insert) Insert(with InsertWith) (*sql.Stmt, error) {
stmt, err := with.Prepare(ins.SQL())
if err != nil {
return nil, err
}
defer func(stmt *sql.Stmt) {
_ = stmt.Close()
}(stmt)
_, err = stmt.Exec(ins.Args()...)
return stmt, err
}

// InsertContext prepares and executes a SQL INSERT statement on a *sql.DB, *sql.Tx, *sql.Conn,
// or other Inserter-compatible interface to PrepareContext and ExecContext.
func (ins *Insert) InsertContext(ctx context.Context, with InsertWith) (*sql.Stmt, error) {
stmt, err := with.Prepare(ins.SQL())
if err != nil {
return nil, err
}
defer func(stmt *sql.Stmt) {
_ = stmt.Close()
}(stmt)
_, err = stmt.ExecContext(ctx, ins.Args()...)
return stmt, err
}
Loading

0 comments on commit f27b2f9

Please sign in to comment.