- template, use ByteBuffer[] for performance tuning?
- web: get/form post, validate bean class and code generation for param serialization?
- general retry and throttling?
- webservice: client retry on network issue?
- website static content security check, (in server env, this is handled by nginx directly)
- validator: annotation for website, like @HTMLSafeString?
- cm: config management, dynamic update properties?
- cache: advanced feature: local/remote 2 level, async get on expiration, pre warm?
- template security check, escaping and etc
- http: make http session https only?
- think about /test//b, and /:path(*)
- framework error, queue listener, background task error notification?
- extend i18n message to validator/format/called by code directly
- faster synchronous rpc
- httpclient: throw HTTPClientException when status code is not supported
- template: fix template engine with c:src in String template
- validate: support Optional
- validate: removed @ValueNotNull/@ValueNotEmpty/@ValuePattern, use @NotNull/@NotEmpty/@Pattern to simplify
- mongo: count() supports null filter
- background-task: fix task can be broken if exception occurred
- error: fix errorCode.severity handling by remoteServiceException
- mongo: support mapReduce
- web: support return all values for file upload and form post
- web: support PUT for file uploading
- http: fixed pass errorCode.severity
- http: add Map<String, String) queryParams();
- template: remove c:msg, use m:text instead
- exception: removed @Warning, use programmable severity (to simplify webservice exception handling and error report)
- template: support m:text and m:{attr} for message body and attribute
- route: fix dynamic path should not match trailing slash
- error: refactory errorCode support, make webservice client translate
- httpclient: put error code to httpclientexception
- rabbitmq: fixed queue listener to report error code on ack
- thread: try to shutdown thread pool gracefully
- template: hide templateManager, not needed by application
- mongo: log Bson filter/projection/sort in json string format
- json: support Optional as field, queue/ws interface supported too
- http: update httpclient request to accept ContentType
- es: update to 2.3.1
- httpclient: throw HTTPClientException for invalid url
- log: put 1M limit of trace log to forward to remote
- log: add message filter support
- mongo: support eval for db-migration
- mongo: set connection max idle time to 30mins
- hmac: changed input from String to bytes
- es: update to 2.3.0
- validate: add @ValuePattern
- http: request supports body() method
- http: added 422 status code
- log: actionLogContext.put checks duplication to avoid huge trace log
- es: update es to 2.2.1
- web: for file response, close exchange when end
- mongo: update default timeout to 15s
- rabbitmq: publisher supports message with priority
- http: added status code 410
- build: update gradle 2.12, flyway 4.0
- mongo: support @MongoEnumValue
- db: renamed @EnumValue to @DBEnumValue
- template: remove LanguageProvider as first step for i18n message support refactory
- stopwatch: start to use nanoTime
- log: actionlog start to trace cpuTime
- mongo: register enum codec from entity, for filter
- http: support statusCode 429
- rabbitmq: update to 3.6.1
- rabbitmq: make rabbitmq listener use dedicated thread pool
- executor: remove Batch support
- scheduler: use dedicated thread pool
- search: remove client(), add analyze/index meta support
- mongo: added Mongo interface for unit test and management
- validation: support @Pattern
- mongo: disable cursorFinalizer, framework always close cursor
- thread: limit executor to processor*4 thread, to collect data for next phase design
- config: update template and message config API
- web: support convert query param as value type
- template: refine i18n language support
- mongo: removed old Mongo interface, refined mongo impl
- mongo: add timeout to query, support MongoCollection
- mongo: updated to 3.2.2, added fongo support back
- search: exposed ElasticSearch as interface
- queue: tweak queue listener to result in less thread when rabbitmq is busy, track slow acknowledge
- batch: allow specify maxConcurrentHandlers for batch
- async: provide thread name for all thread pool
- redis: update default redis timeout to 5s
- validation: update validation exception message
- json: add from/to enum value to JSON, to support search query build with enum
- util: ASCII supports char
- log: improve log error handling
- fix: allow logger to log null message, e.g. errorMessage of NullPointerException
- rabbitmq: unified single/batch message polling
- util: moved InputStreams to util, open to use
- log: make max trace log 3000, and only append warn/error after
- search: added SearchResponse to convert hits to object on framework level
- tuning: internal tuning, move low level optimization class to impl package, leave core.framework.util simple ones
- httpclient: use byte[] as body, remove ByteBuf
- log: use 500k as max trace log to forward to log processor, removed max trace event size limit, truncate after 5000 and only add warning events
- cache: make cache process byte[] directly to redis
- log: limit trace log 200k max per line, rise max log events per action to 10,000
- json: add afterburner module for object binding performance
- db: lower default db timeout to 15s
- queue: updated rabbitMQ api and config api
- json: start convert json to bytes directly, to lower memory footprint with queue/ES/cache
- web: removed web/not-found, web/method-not-allowed action assign, since we use error_code now
- background: moved pool-cleanup, collect-stat job to background thread, not included in action
- web: renamed all internal /management/ path to /_sys/
- monitor: initial monitoring draft, forward monitor metrics via logforwarder
- template: invalid url attr will write src="", container will write empty if content is null
- elasticsearch: support 2.2.0, load groovy plugin in test context
- redis: loose slow_redis warning threshold and timeout, on busy server due to CPU context switch, it's relative easy to hit it
- web: requestURL now contains QueryString, (requestURL is url without decoding)
- log: renamed all slow query error_code and naming
- pool: add error code POOL_TIME_OUT
- template: warning if url fragment gets null url
- log: forward log one by one to simplify, batch happens on log-processor
- log: make 3rd party log level to info, (e.g. ES log sampler error by INFO level in separated thread)
- elasticsearch: set ping timeout, support dynamic index name (for alias or time serial index name)
- fix: typo in @NotEmpty/@ValueNotEmpty
- ws: support enum in path param
- cache: update getAll to return Map<String, T>
- web: removed URIBuilder, added Encodings.encodeURIComponent and decodeURIComponent
- web: use URI query param encoding/decoding to set/get cookie (refered as URLEncoding in other place, e.g. jquery)
- schedule: support weekly and monthly trigger
- bytebuf: improve for skip/available as it will be wrapped by buffered stream or S3 client
- web: mark path not found error code to PATH_NOT_FOUND (to ignore 3rd party scan)
- batch: new async support, use Executor
- mongo: support LocalDateTime in filter
- httpclient: fix NPE with HttpEntity is null on 204
- elasticsearch: ignore cluster name for transport client
- url: encode '+' for path segment to keep compatible with other impl, e.g. undertow, AWS S3
- mongo: update driver to 3.2.0
- quality: support jacoco report
- lib: updated undertow to 1.3.10, jackson to 2.6.4, update quality check lib
- db: added repository.select(query) to support complex query + limit
- log: dynamic group action log message to forward, 2000 or 5M which comes first
- log: update log forwarding structure, send in batch
- log: moved ErrorCode to core.framework.api.log.ErrorCode
- elasticsearch: update to 2.1.0
- log: restructure trace/warning log, for prod log aggregation
- hash: added md5 support
- queue: removed sns/sqs support, only use rabbitmq from now on
- redis/cache: mset with expiration, update jedis to 2.8.0
- redis: fix the typo in log,
- bug: fix cache getAll/setAll
- http: added ACCEPTED(202) for async response code
- db: removed selectInt/selectLong/selectString, use selectOne instead with target view class, e.g. selectOne(sql, String.class, params)
- benchmark: added jmh for systemically performance turning
- util: added ASCII to handle ASCII chars
- web: validate empty pathParam on both server and client side
- util: fast impl of Strings.split(char)
- uri: simplify URIBuilder to match our use cases and performance tuning
- util: ByteBuf put byte[]
- refactory: performance tuning for html/uri encode/decode
- bug: fix html parser/lexer process emtpty end script tag.
- redis: update redis mget to return Map<String,String>
- route: validate path segment and variable
- bug: fixed QueryParam encoding to encode +/?/=
- bug: fixed URLFragment.isValidURL to allow '%'
- util: enhanced ByteBuf to make it can be used in broader scenario.
- encodings: removed hex encoding, we don't have use case any more, use base64 instead
- encodings: removed url encoding from Encodings, added URIBuilder
- cache: support get multiple keys in batch
- cache: changed get(Supplier) to get(Function<String,T>) to keep consistent with getAll, and be more functional style
- bug: fix http content type parsing multipart
- search: support bulk index
- yml: removed yml support, use JSON instead
- mongo: removed fongo, updated mongo driver to 3.1.0
- mongo: entity validation, not allow mix with JAXB
- template: refactory html parser validation, validate boolean attribute
- template: template must register in config before using, enforce safety
- template: validate empty attribute
- template: refactory html parser, html validation, void element
- config: support sys.http.port
- template: support i18n message with hierarchy, "en_US" fall back to "en"
- template: support dynamic href
- mongo: added get(id) method to replace findOne()
- mongo: added Query to support complex query
- search: update to elasticsearch 2.0
- lib: update undertow to 1.3.4, jackson to 2.6.3
- http: introduced strong typed ContentType
- template: validate cdn attribute, url must start with /
- template: c:msg="" use key directly, removed expression support
- template: built-in i18n support
- properties: read in utf-8
- httpclient: fix text() to use utf-8 as encoding
- webservice: unified web service call error flow, RemoteServiceException provided structural info to be handled in client service app
- exception: provide ErrorCode support for error flow
- template: removed c:cdn, automatically replace all src/href applies
- template: new template syntax, only support HTML template
- rabbitmq: publisher mark pool item to be broken if AlreadyClosedException (throws when exchange/queue is not configured correctly in RabbitMQ)
- site: exposed WebDirectory to injection context to provide way let website access web directory
- mongo: support assigned id
- mongo: config support
- lib: removed hsql to default test-compile scope, not all services had db.
- gradle: core-ng app support ant property filtering, conf/env/web content override
- test: rewrite env conf validator, to verify /conf/env/resources and /conf/env/web
- web: #cdn function append version param
- gradle: checkstyle update to 6.11.2, finally support intention for lambda
- web: static content supports file under root directly (like robot.txt and favicon.ico)
- web: static content removed 304 support, because static content is always handled by nginx in server env.
- web: update undertow to 1.3.0.Final
- db: improve setParam error message
- util: Files, added common functions
- test: refactory test dependency and structure, support api().client() mock
- web: support CDN config and #cdn() function, in sys.properties uses sys.cdn.host=
- template: expression validate return type, for for/if
- validate: added @NotEmpty (for string), @ValueNotEmpty, @ValueNotNull (for collection)
- template: print location when expression failed
- template: removed #if(condition?a:b), always prefer to write method in model
- template: support single quote string
- template: add #if as built-in method for inline condition
- bug: fixed actionLog result is not updated
- bug: fixed web/IO allow content-length = 0, and catch all errors during IO in order to generate action log
- util: added ByteBuf for high performance network IO and NIO
- http-client: response bytes() changed to inputStream()
- web-service: moved core.framework.api.web.client.WebServiceRequestSigner to core.framework.api.web.service.WebServiceRequestSigner
- web: changed form parsing threading model, support file upload
- web: read json body in IO thread
- refactory: web-service code generator, logger
- config: better error message for load not found property
- lib: update all 3rd party lib up to date
- web-service: fix web service GET query, not append param if null
- log-forward: clear up queue if rabbitmq is down
- log: support to disable action/trace, !!! sys.properties, use sys.log.actionLogPath=console, sys.log.traceLogPath=console in dev env.
- queue: config, publish(String[] destinations) will call publish(String destination) if length = 1
- log: to protect log ES, not forward trace log if lines is more than max hold size = 5000, the log will still be written to file and application is responsible to split task into smaller chunk
- batch: tweak async executor for batch process
- redis: refined config, removed name support and added "sys.redis.host"
- search: config support, annotation support
- log-processor: due to search support changed, log processor must match latest core-ng
- validate: type validator disallow cycle reference data structure (include same node as parent/child)
- redis: config support
- redis: added setIfAbsent, use SET to replace SETEX
- search: removed parent/child support, always use nested from now on
- search: support parent/child
- search: added delete and other tuning
- bug: fixed DBConfig to handle multiple DB with different name
- search: added get()
- web: web bean validation, disallow to use @XmlEnum(Integer.class) on enum class, (in this case, not use enum, just use Integer, this is due to incomplete Jackson JAXB support)
- bug: fixed when rabbitmq server is down, the rabbitmq listener/log forward may keep trying without wait, cause cpu 100%
- httpClient: reduced default timeout to 1 min
- httpClient: disable cookie and auth by default
- util: InputStreams use 8K as buffer if length can not be determined, optimize for HTTPClient download with gzip
- tuning: various of tunings, utils/IO/random and etc
- util: removed Asserts, not really needed, in actual business code, just throw error/validation exception
- db: removed RowMapper, use view instead (broken change!)
- web: enum must have @XmlEnumValue
- config: renamed AbstractApplication to App (broken change!)
- db: removed repository select where clause must contains '?', exception is like "where some_column is null";
- db: removed Query, prefer use sql + param... (broken change!)
- web: updated Request.host() to Request.hostName(), according to url standard: http://bl.ocks.org/abernier/3070589
- db: replace repository insert/update query with dynamic code generation
- db: use @EnumValue to map the enum value in db (broken change!)
- db: lower the slow query to 5s and too many results to 1000
- redis: raise slow query to 200ms, (considering GC and network latency)
- redis: removed keys() (use SCAN in future if needed)
- redis: use binaryJedis and other changes according to profiling
- db: fix selectInt/String with null returned
- search: validate search index document object
- validator: partial validate for update (ignore notNull)
- db: validate for update
- validation: added @Min/@Max
- db: removed db c3p0 pool, use internal pool
- sysmodule: removed sys.jdbc.pool.minSize/maxSize, all env should be same, and if need to specify, put to App
- rabbitmq: check slow query
- rabbitmq: make listener use native thread
- redis: use internal pooling
- rabbitmq: channel pooling (send message perf is 20x faster if not closing channel)
- web: assign action for 404 => web/not-found, 405 => web/method-not-allowed
- scheduler: make scheduler for both internal and external scheduling
- db: use label not column name to map view
- FIX: make ControllerInspector works with jdk 1.8.0_60
- rabbitmq: handle ShutdownSignalException gracefully
- lib: updated jackson/undertow/rabbitmq/httpclient lib
- FIX: ControllerInspector does not work with jdk 1.8.0_60, disable temporarily
- log: refactor/tuning impl
- log: update log-processor ES mappings
- ES: API changed to provide more flexibility to operate index and type
- log: draft of trace log forwarding
- http: fix put/post without content-type
- http server: interceptor ignore built-in controllers
- mongo: use LinkedHashMap to keep same order as in mongo
- log: draft of log forwarding
- web: fix webservice controller inspection (getAnnotation/methodInfo)
- mongo: find supports orderby/skip/limit
- log: push action log to rabbitmq, index by ES/Kibana
- log: renamed action log requestId to id, and use refId for reference
- webService: WebContext can retrive request() to support web service
- rabbitmq: publish message with appId = -Dcore.appName
- webservice client: pass -Dcore.appName via "client" header
- internal: expose web template manager for cms widget impl
- internal: renamed -Dcore.web to -Dcore.webPath, added -Dcore.appName, prepare for log aggregating
- make default values to fit AWS medium/large instance, simplify env properties
- rabbitmq: make default user/password to rabbitmq/rabbitmq
- redis: make default pool size to (5,50)
- db: make default pool size to (5,50)
- cache/session: update default redis pool size, (min=8,max=32) optimized to AWS medium/large instances.
- redis: print pool size info for slow query
-
updated c3p0 to 0.9.5.1 (all client must update c3p0 lib to use this version of core-ng)
make c3p0 not use thread pool for checkin, we don't do test on checkin, this improves performance under high load
dataSource.setForceSynchronousCheckins(true);
-
updated mysql driver to 5.1.36 for db-migration
- website: message support, site().message().messageProvider() is for custom message service
- mongo: refactory, support fongo, minor API changes
- queue: composite queue publisher
- queue: MessagePublisher.publish with routingKey, only support for RabbitMQ
- template: validate model class
- add TemplateEngine for general purpose
- module: renamed bind(supplier) to bindSupplier()
- template: "include" support
- queue: renamed MessagePublisher.reply to publish
- template: custom function support, prepare for #msg, #js, #css
- template: load from string
- db repository, added selectAll()
- template engine first draft impl, removed thymeleaf
- website static content
- refactory rabbitmq support
- update elasticsearch to 1.7
- standardized validation exception and make validator throw validationException, refactory validator
- fix: http response body validation pass empty list
- databaseImpl, track available/total connections when getting conn from pool
- httpClient, allow server ssl cert change during renegotiation
- tune http client log info
- enhanced Files, create temp file, logging
- fix: api() client to pass "Accept: application/json"
- replace URL/URLPath encoding with apache common codec
- tuned async task begin/end log message
- measure startup time
- measure time on test db schema creation
- update gradle to 2.5
- update build gradle to publish to s3 directly
- fix: api webservice client encode path param
- renamed StandardAppModule to SystemModule, and added jdbc pool properties
- fix: requestURL(), parse x-forwarded-port to get requested port
- fix: test hsql db map BigDecimal to DECIMAL(10,2)
- refactory config structure to simplify test context
- refactory class validation into general one with spec
- WebContext to pass from interceptor
- updated web customErrorHandler, moved it to http() and in top level of Module
- add requiredProperty() method in module
- async task support, inject AsyncExecutor
- webservice support List as request/response, for channel-advisor and also for us
- removed module.loadYML(), use YAML.load(ClasspathResources.text()) instead, for db init data, use YAML loadlist then call repository.batchInsert
- improve web service interface validator, better error message for bean param/path param missing @PathParam
- Cache management controllers (list names/clear key)
- move repository() into db() and register db entity class to view.
- fix: webservice interface validator should allow String as @PathParam
- db added timeout for checkout conn and query, default is 60s
- update ElasticSearch API according to ES 1.6.0
- bind with supplier to support for expensive or external dependent object, test context can override by not calling supplier.get()
- mark trace in ActionLogContext, write trace log for all subsequent actions (apiCall/messages)
- create Application.configure()/start(), to make core-ng be used as script for testing/scripting
- trace external calling time and count, e.g. db, redis
- update trace log path
- cache value type validator
- refactory cache impl
- Cache error handle, fault tolerant
- database batchInsert/Delete
- httpClient supports download bytes
- webservice client
- support test initDB(), runscript and createSchema
- update rabbitmq/db config
- refactory validator
- refactory module/config(testConfig)/builder/impl structure
- refactory session provider, local session cleanup
- added rabbitmq subscriber and listener
- refactory module config, session changes to site().session()
- added core-ng-api for interface module
- updated row mapper, use code generation, updated Row interface
- added core-ng-test support
- YML supports type
- support to add cache without name
- cache management controller
- add repository support