Change Drop Table Method(dropTableIfExists) #346

dongyeon94 · 2024-08-29T01:59:44Z

fix if exists command not support error

supportsTableIfExistsClause() method always true

so my system print out this error

2024-08-29 10:13:48.061 +0900 [INFO] (0001:transaction): SQL: DROP TABLE IF EXISTS "CLAIM_INFO" 2024-08-29 10:13:48.065 +0900 [ERROR] (0001:transaction): Operation failed (first exception:{SQLState=933, ErrorCode=42000}) java.sql.SQLSyntaxErrorException: ORA-00933: SQL command not properly ended at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:450) at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:399) at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:1059) at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:522)

…) method is always true)

dmikurube · 2024-08-29T03:03:37Z

Are we/you sure that the catch (Exception ex) {} case is always IF EXISTS is not supported? Doesn't it have any other possible error case? Is that error message appropriate for any case?
What was your actual error case? What RDBMS? Version?
supportsTableIfExistsClause() is expected to be implemented by inheritance. For example, embulk-output-oracle had overridded this to return false;
- https://github.com/embulk/embulk-output-jdbc/blob/v0.10.1/embulk-output-oracle/src/main/java/org/embulk/output/oracle/OracleOutputConnection.java#L65-L69
- We've terminated to maintain embulk-output-oracle as we do not have a testing environment for Oracle DB, though.

dongyeon94 · 2024-08-29T04:20:31Z

our RDMS version is

oracle 11.2

our embulk and jruby version

embulk version 0.11.1
jruby version 9.4.6.0

we try use "embulk-output-oracle" and we get error 'OutputPlugin 'oracle' is not found'

I thought this error was because not supporting oracle, so I changed it to JDBC plugin and test it
and try embulk-output-jdbc result

jdbc plugin not support oracle RDMS?

we are already install

embulk-input-oracle (0.9.0)
embulk-output-oracle (0.8.7)
oracle (0.0.1)
embulk-input-jdbc (0.13.2 java)
embulk-output-jdbc (0.10.6 java)

dmikurube · 2024-08-29T04:33:46Z

Hmm, then the oracle plugin should work, but I understand it may fail as it's not updated recently.

Can you share the "full" exception output? The text format would be more preferred than a screenshot. Also nice to be in the "code format" (``` 〜 ```).

Cc: @hito4t

dongyeon94 · 2024-08-29T04:44:14Z

full error text is


2024-08-29 13:09:20.849 +0900 [INFO] (main): Loaded JRuby runtime 9.4.6.0
2024-08-29 13:09:22.640 +0900 [INFO] (main): Started Embulk v0.11.1
2024-08-29 13:09:22.825 +0900 [INFO] (0001:transaction): Embulk system property "plugins.input.jdbc" is not set.
2024-08-29 13:09:22.825 +0900 [INFO] (0001:transaction): Embulk system property "plugins.default.input.jdbc" is not set.
2024-08-29 13:09:25.974 +0900 [INFO] (0001:transaction): Gem's home and path are set by system configs "gem_home": "/var/digdag_home/.embulk/lib/gems", "gem_path": ""
2024-08-29 13:09:27.283 +0900 [INFO] (0001:transaction): Loaded JRuby runtime 9.4.6.0
2024-08-29 13:09:27.379 +0900 [INFO] (0001:transaction): Loaded plugin embulk-input-jdbc (0.13.2)
2024-08-29 13:09:27.497 +0900 [INFO] (0001:transaction): Embulk system property "plugins.output.oracle" is not set.
2024-08-29 13:09:27.497 +0900 [INFO] (0001:transaction): Embulk system property "plugins.default.output.oracle" is not set.
2024-08-29 13:09:27.584 +0900 [INFO] (0001:transaction): Loaded plugin embulk-output-oracle (0.8.7)
org.embulk.config.ConfigException: OutputPlugin 'oracle' is not found.
com/google/common/base/Supplier
	at org.embulk.plugin.PluginManager.buildPluginNotFoundException(PluginManager.java:155)
	at org.embulk.plugin.PluginManager.newPlugin(PluginManager.java:99)
	at org.embulk.spi.ExecSessionInternal.newPlugin(ExecSessionInternal.java:358)
	at org.embulk.spi.ExecInternal.newPlugin(ExecInternal.java:50)
	at org.embulk.exec.BulkLoader$ProcessPluginSet.<init>(BulkLoader.java:442)
	at org.embulk.exec.BulkLoader.doRun(BulkLoader.java:516)
	at org.embulk.exec.BulkLoader.access$000(BulkLoader.java:36)
	at org.embulk.exec.BulkLoader$1.run(BulkLoader.java:353)
	at org.embulk.exec.BulkLoader$1.run(BulkLoader.java:350)
	at org.embulk.spi.ExecInternal.doWith(ExecInternal.java:26)
	at org.embulk.exec.BulkLoader.run(BulkLoader.java:350)
	at org.embulk.EmbulkEmbed.run(EmbulkEmbed.java:278)
	at org.embulk.EmbulkRunner.runInternal(EmbulkRunner.java:288)
	at org.embulk.EmbulkRunner.run(EmbulkRunner.java:153)
	at org.embulk.cli.EmbulkRun.runInternal(EmbulkRun.java:108)
	at org.embulk.cli.EmbulkRun.run(EmbulkRun.java:24)
	at org.embulk.cli.Main.main(Main.java:53)
	Suppressed: org.embulk.plugin.PluginSourceNotMatchException
		at org.embulk.plugin.BuiltinPluginSource.newPlugin(BuiltinPluginSource.java:244)
		at org.embulk.plugin.PluginManager.newPlugin(PluginManager.java:76)
		... 15 more
	Suppressed: org.embulk.plugin.PluginSourceNotMatchException
		at org.embulk.plugin.maven.MavenPluginRegistry.findPluginPaths(MavenPluginRegistry.java:143)
		at org.embulk.plugin.maven.MavenPluginRegistry.lookup(MavenPluginRegistry.java:63)
		at org.embulk.plugin.maven.MavenPluginSource.newPlugin(MavenPluginSource.java:33)
		at org.embulk.plugin.PluginManager.newPlugin(PluginManager.java:82)
		... 15 more
	Suppressed: org.embulk.plugin.PluginSourceNotMatchException
		at org.embulk.plugin.SelfContainedPluginRegistry.search(SelfContainedPluginRegistry.java:77)
		at org.embulk.plugin.SelfContainedPluginRegistry.lookup(SelfContainedPluginRegistry.java:64)
		at org.embulk.plugin.SelfContainedPluginSource.newPlugin(SelfContainedPluginSource.java:44)
		at org.embulk.plugin.PluginManager.newPlugin(PluginManager.java:88)
		... 15 more
	Suppressed: org.embulk.plugin.PluginSourceNotMatchException: Failed to initialize embulk-output-oracle.
		at org.embulk.jruby.JRubyPluginSource.newPlugin(JRubyPluginSource.java:89)
		at org.embulk.plugin.PluginManager.newPlugin(PluginManager.java:94)
		... 15 more
	Caused by: java.lang.NoClassDefFoundError: com/google/common/base/Supplier
		at java.lang.Class.getDeclaredConstructors0(Native Method)
		at java.lang.Class.privateGetDeclaredConstructors(java/lang/Class.java:2671)
		at java.lang.Class.getConstructor0(java/lang/Class.java:3075)
		at java.lang.Class.getConstructor(java/lang/Class.java:1825)
		at org.embulk.plugin.PluginManager.getConstructorWithProperties(org/embulk/plugin/PluginManager.java:130)
		at org.embulk.plugin.PluginManager.newPluginInstance(org/embulk/plugin/PluginManager.java:106)
		at java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:498)
		at org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(org/jruby/javasupport/JavaMethod.java:330)
		at org.jruby.javasupport.JavaMethod.invokeStaticDirect(org/jruby/javasupport/JavaMethod.java:232)
		at RUBY.new_java(/var/digdag_home/.embulk/lib/gems/gems/embulk-0.11.1-java/lib/embulk/java_plugin.rb:92)
		at RUBY.new_java_output(/var/digdag_home/.embulk/lib/gems/gems/embulk-0.11.1-java/lib/embulk/plugin.rb:159)
		at RUBY.new_java_output(uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/forwardable.rb:238)
		at org.jruby.RubyClass.finvokeWithRefinements(org/jruby/RubyClass.java:540)
		at org.jruby.RubyClass.finvoke(org/jruby/RubyClass.java:528)
		at org.jruby.embed.internal.EmbedRubyObjectAdapterImpl.doInvokeMethod(org/jruby/embed/internal/EmbedRubyObjectAdapterImpl.java:240)
		at org.jruby.embed.internal.EmbedRubyObjectAdapterImpl.callMethod(org/jruby/embed/internal/EmbedRubyObjectAdapterImpl.java:156)
		at org.jruby.embed.ScriptingContainer.callMethod(org/jruby/embed/ScriptingContainer.java:1448)
		at java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:498)
		at org.embulk.jruby.ScriptingContainerDelegateImpl.callMethod(org/embulk/jruby/ScriptingContainerDelegateImpl.java:644)
		at org.embulk.jruby.LazyScriptingContainerDelegate.callMethod(org/embulk/jruby/LazyScriptingContainerDelegate.java:230)
		at org.embulk.jruby.JRubyPluginSource.newPlugin(org/embulk/jruby/JRubyPluginSource.java:87)
		at org.embulk.plugin.PluginManager.newPlugin(org/embulk/plugin/PluginManager.java:94)
		at org.embulk.spi.ExecSessionInternal.newPlugin(org/embulk/spi/ExecSessionInternal.java:358)
		at org.embulk.spi.ExecInternal.newPlugin(org/embulk/spi/ExecInternal.java:50)
		at org.embulk.exec.BulkLoader$ProcessPluginSet.<init>(org/embulk/exec/BulkLoader.java:442)
		at org.embulk.exec.BulkLoader.doRun(org/embulk/exec/BulkLoader.java:516)
		at org.embulk.exec.BulkLoader.access$000(org/embulk/exec/BulkLoader.java:36)
		at org.embulk.exec.BulkLoader$1.run(org/embulk/exec/BulkLoader.java:353)
		at org.embulk.exec.BulkLoader$1.run(org/embulk/exec/BulkLoader.java:350)
		at org.embulk.spi.ExecInternal.doWith(org/embulk/spi/ExecInternal.java:26)
		at org.embulk.exec.BulkLoader.run(org/embulk/exec/BulkLoader.java:350)
		at org.embulk.EmbulkEmbed.run(org/embulk/EmbulkEmbed.java:278)
		at org.embulk.EmbulkRunner.runInternal(org/embulk/EmbulkRunner.java:288)
		at org.embulk.EmbulkRunner.run(org/embulk/EmbulkRunner.java:153)
		at org.embulk.cli.EmbulkRun.runInternal(org/embulk/cli/EmbulkRun.java:108)
		at org.embulk.cli.EmbulkRun.run(org/embulk/cli/EmbulkRun.java:24)
		at org.embulk.cli.Main.main(org/embulk/cli/Main.java:53)
	Caused by: java.lang.ClassNotFoundException: com.google.common.base.Supplier
		at org.embulk.plugin.PluginClassLoader.loadClass(PluginClassLoader.java:160)
		at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
		at java.lang.Class.getDeclaredConstructors0(Native Method)
		at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
		at java.lang.Class.getConstructor0(Class.java:3075)
		at java.lang.Class.getConstructor(Class.java:1825)
		at org.embulk.plugin.PluginManager.getConstructorWithProperties(PluginManager.java:130)
		at org.embulk.plugin.PluginManager.newPluginInstance(PluginManager.java:106)
		at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
		at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
		at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
		at java.lang.reflect.Method.invoke(Method.java:498)
		at org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:330)
		at org.jruby.javasupport.JavaMethod.invokeStaticDirect(JavaMethod.java:232)
		at org.jruby.java.invokers.StaticMethodInvoker.call(StaticMethodInvoker.java:53)
		at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:291)
		at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:324)
		at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:66)
		at org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:76)
		at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:164)
		at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:151)
		at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:212)
		at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:456)
		at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:195)
		at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:346)
		at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:66)
		at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:128)
		at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:115)
		at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:446)
		at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:92)
		at org.jruby.ir.instructions.CallBase.interpret(CallBase.java:548)
		at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:363)
		at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:66)
		at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:128)
		at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:115)
		at org.jruby.RubyClass.finvokeWithRefinements(RubyClass.java:540)
		at org.jruby.RubyClass.finvoke(RubyClass.java:528)
		at org.jruby.runtime.Helpers.invoke(Helpers.java:703)
		at org.jruby.embed.internal.EmbedRubyObjectAdapterImpl.doInvokeMethod(EmbedRubyObjectAdapterImpl.java:240)
		at org.jruby.embed.internal.EmbedRubyObjectAdapterImpl.callMethod(EmbedRubyObjectAdapterImpl.java:156)
		at org.jruby.embed.ScriptingContainer.callMethod(ScriptingContainer.java:1448)
		at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
		at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
		at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
		at java.lang.reflect.Method.invoke(Method.java:498)
		at org.embulk.jruby.ScriptingContainerDelegateImpl.callMethod(ScriptingContainerDelegateImpl.java:644)
		at org.embulk.jruby.LazyScriptingContainerDelegate.callMethod(LazyScriptingContainerDelegate.java:230)
		at org.embulk.jruby.JRubyPluginSource.newPlugin(JRubyPluginSource.java:87)
		at org.embulk.plugin.PluginManager.newPlugin(PluginManager.java:94)
		at org.embulk.spi.ExecSessionInternal.newPlugin(ExecSessionInternal.java:358)
		at org.embulk.spi.ExecInternal.newPlugin(ExecInternal.java:50)
		at org.embulk.exec.BulkLoader$ProcessPluginSet.<init>(BulkLoader.java:442)
		at org.embulk.exec.BulkLoader.doRun(BulkLoader.java:516)
		at org.embulk.exec.BulkLoader.access$000(BulkLoader.java:36)
		at org.embulk.exec.BulkLoader$1.run(BulkLoader.java:353)
		at org.embulk.exec.BulkLoader$1.run(BulkLoader.java:350)
		at org.embulk.spi.ExecInternal.doWith(ExecInternal.java:26)
		at org.embulk.exec.BulkLoader.run(BulkLoader.java:350)
		at org.embulk.EmbulkEmbed.run(EmbulkEmbed.java:278)
		at org.embulk.EmbulkRunner.runInternal(EmbulkRunner.java:288)
		at org.embulk.EmbulkRunner.run(EmbulkRunner.java:153)
		at org.embulk.cli.EmbulkRun.runInternal(EmbulkRun.java:108)
		at org.embulk.cli.EmbulkRun.run(EmbulkRun.java:24)
		at org.embulk.cli.Main.main(Main.java:53)

Error: OutputPlugin 'oracle' is not found.

In our opinion, the plugin can not find the Oracle package

hiroyuki-sato · 2024-08-29T06:57:47Z

As far as I know, embulk-output-oracle does not support Embulk v0.10 ~ yet.
Maybe, You can run embulk-output-oracle using embulk 0.9.24 as a workaround.

dmikurube · 2024-08-29T07:10:54Z

Ah, yeah, not the Oracle JDBC driver, but com.google.common.base.Supplier was required, which is no longer included since Embulk v0.10.3X.

dongyeon94 · 2024-09-03T04:01:23Z

we are upgrading embulk version 0.9.25 -> 0.11.1

We know that version 11.1 is a stable version, so we are upgrading the version.
Do you have any plan to upgrade v0.10 "embulk-output-oracle" or later??

dmikurube · 2024-09-03T04:16:30Z

As of #302, we (the Embulk project) no longer maintain embulk-output-oracle as we have no testing environment for Oracle.

If you're interested in maintaining it by yourself, or joining other efforts, you may want to talk to @hito4t

dongyeon94 · 2024-09-04T07:18:24Z

@hito4t
How can I joining embulk-output-oracle project?

hito4t · 2024-09-06T08:00:59Z

@dongyeon94
Thank you for the proposal.

I created the repository for embulk-input-oracle (https://github.com/hito4t/embulk-input-jdbc-external), but not yet for embulk-output-oracle.
I'll create the repository from the latest (although not maintained...) source code for embulk-output-oracle.

Please wait for a while.

dongyeon94 · 2024-09-09T01:01:13Z

I See, �thank you.
I'll personally customize it and use it.
Please contact me when we proceed later.

hito4t · 2024-09-10T00:12:32Z

@dongyeon94 @hiroyuki-sato
I've created the repository for embulk-output-oracle, although it doesn't support the latest Embulk yet.
https://github.com/hito4t/embulk-output-jdbc-external

fix if exists command not support error (supportsTableIfExistsClause(…

1526c6d

…) method is always true)

dongyeon94 requested a review from a team as a code owner August 29, 2024 01:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Change Drop Table Method(dropTableIfExists) #346

Change Drop Table Method(dropTableIfExists) #346

dongyeon94 commented Aug 29, 2024

dmikurube commented Aug 29, 2024

dongyeon94 commented Aug 29, 2024 •

edited

Loading

dmikurube commented Aug 29, 2024

dongyeon94 commented Aug 29, 2024

hiroyuki-sato commented Aug 29, 2024

dmikurube commented Aug 29, 2024

dongyeon94 commented Sep 3, 2024

dmikurube commented Sep 3, 2024

dongyeon94 commented Sep 4, 2024

hito4t commented Sep 6, 2024

dongyeon94 commented Sep 9, 2024

hito4t commented Sep 10, 2024

Change Drop Table Method(dropTableIfExists) #346

Are you sure you want to change the base?

Change Drop Table Method(dropTableIfExists) #346

Conversation

dongyeon94 commented Aug 29, 2024

dmikurube commented Aug 29, 2024

dongyeon94 commented Aug 29, 2024 • edited Loading

dmikurube commented Aug 29, 2024

dongyeon94 commented Aug 29, 2024

hiroyuki-sato commented Aug 29, 2024

dmikurube commented Aug 29, 2024

dongyeon94 commented Sep 3, 2024

dmikurube commented Sep 3, 2024

dongyeon94 commented Sep 4, 2024

hito4t commented Sep 6, 2024

dongyeon94 commented Sep 9, 2024

hito4t commented Sep 10, 2024

dongyeon94 commented Aug 29, 2024 •

edited

Loading