[Bug] DuplicateKeyException encountered while retrieving the job execution history

[arshadmohammad]

immediately after task execution start, task instance page is opened. For small jobs it is throwing following errors

2025-01-29 12:06:18.800 seatunnel WM-TVJM7KC4Y4 ERROR [tr:,sp:] [qtp431300939-38] [GlobalExceptionHandler.logError():78] -

Error updating database. Cause: java.sql.SQLIntegrityConstraintViolationException: Duplicate entry '16473878314272' for key 't_st_job_instance_history.PRIMARY'

The error may exist in org/apache/seatunnel/app/dal/mapper/JobInstanceHistoryMapper.java (best guess)

The error may involve org.apache.seatunnel.app.dal.mapper.JobInstanceHistoryMapper.insert-Inline

The error occurred while setting parameters

SQL: INSERT INTO t_st_job_instance_history ( id, dag ) VALUES ( ?, ? )

Cause: java.sql.SQLIntegrityConstraintViolationException: Duplicate entry '16473878314272' for key 't_st_job_instance_history.PRIMARY'

; Duplicate entry '16473878314272' for key 't_st_job_instance_history.PRIMARY'; nested exception is java.sql.SQLIntegrityConstraintViolationException: Duplicate entry '16473878314272' for key 't_st_job_instance_history.PRIMARY'
org.springframework.dao.DuplicateKeyException:

Error updating database. Cause: java.sql.SQLIntegrityConstraintViolationException: Duplicate entry '16473878314272' for key 't_st_job_instance_history.PRIMARY'

The error may exist in org/apache/seatunnel/app/dal/mapper/JobInstanceHistoryMapper.java (best guess)

The error may involve org.apache.seatunnel.app.dal.mapper.JobInstanceHistoryMapper.insert-Inline

The error occurred while setting parameters

SQL: INSERT INTO t_st_job_instance_history ( id, dag ) VALUES ( ?, ? )

Cause: java.sql.SQLIntegrityConstraintViolationException: Duplicate entry '16473878314272' for key 't_st_job_instance_history.PRIMARY'

; Duplicate entry '16473878314272' for key 't_st_job_instance_history.PRIMARY'; nested exception is java.sql.SQLIntegrityConstraintViolationException: Duplicate entry '16473878314272' for key 't_st_job_instance_history.PRIMARY'
at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:247)
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:70)
at org.mybatis.spring.MyBatisExceptionTranslator.translateExceptionIfPossible(MyBatisExceptionTranslator.java:91)
at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:441)
at jdk.proxy2/jdk.proxy2.$Proxy85.insert(Unknown Source)
at org.mybatis.spring.SqlSessionTemplate.insert(SqlSessionTemplate.java:272

[arshadmohammad]

This is happening because the job history insert is being hit from two code flow at the same time.
1:
java.lang.Throwable: insert: at org.apache.seatunnel.app.service.impl.JobMetricsServiceImpl.syncHistoryJobInfoToDb(JobMetricsServiceImpl.java:599) at org.apache.seatunnel.app.service.impl.JobMetricsServiceImpl.syncJobDataToDb(JobMetricsServiceImpl.java:543) at org.apache.seatunnel.app.service.impl.JobInstanceServiceImpl.complete(JobInstanceServiceImpl.java:358) at org.apache.seatunnel.app.service.impl.JobExecutorServiceImpl.waitJobFinish(JobExecutorServiceImpl.java:173)
2:
java.lang.Throwable: insert: at org.apache.seatunnel.app.service.impl.JobMetricsServiceImpl.syncHistoryJobInfoToDb(JobMetricsServiceImpl.java:599) at org.apache.seatunnel.app.service.impl.JobMetricsServiceImpl.getJobDAG(JobMetricsServiceImpl.java:506) at org.apache.seatunnel.app.controller.JobMetricsController.getJobDAG(JobMetricsController.java:66) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

[arshadmohammad]

After this fix job execution is displayed without error