-
Notifications
You must be signed in to change notification settings - Fork 64
If some rows fail because of data issues then that row is skipped and written to an error file <file_name>_err.csv, This file has the error and the original row that failed. The rest of the file is uploaded and processed.
If the number of failures exceeds 10,000 then the entire file is rejected and processing is stopped
If you want to upsert data to a dataset then that dataset must have a UniqueId defined. You can do that by editing the json and setting the attribute ‘isUniqueId=true’ for that field
Only fields of type Text can be set as UniqueId. Only one field can be set as UniqueId The schema of the target dataset must exactly match the schema of the file being loaded.
{
"name" : "MyId",
"fullyQualifiedName" : "MyId",
"label" : "MyId",
"description" : "MyId",
"type" : "Text",
"precision" : 18,
"scale" : 0,
"defaultValue" : "0",
"format" : null,
"isSystemField" : false,
"isUniqueId" : true,
"isMultiValue" : false,
"multiValueSeparator" : null,
"fiscalMonthOffset" : 0,
"firstDayOfWeek" : 1,
"canTruncateValue" : true,
"isSkipped" : false,
"decimalSeparator" : "."
}
* How do I add computed field to the csv
1. 1. First run the upload action for your csv. This will generate the default json for your csv
1. Then Edit the json (use http://www.jsoneditoronline.org/) and at the end of the last field add a new field.
1. Make sure the type of the field is the return type of computation formula
1. In the field set attribute 'isComputedField' to true
1. In the field set attribute 'computedFieldExpression' to the formula you want to use to calculate your field, for example:
`"computedFieldExpression" : "if (prev.get('OpportunityId') != curr.get('OpportunityId')) {0} else {curr.get('Amount') - prev.get('Amount')}"`
The syntax for this Expression is Java Script. In theory you can do any calculations that you can do in standard java script engine
Complete Field definition from above example:
{
"name" : "AmountDelta",
"fullyQualifiedName" : "AmountDelta",
"label" : "AmountDelta",
"description" : "AmountDelta",
"type" : "Numeric",
"precision" : 18,
"scale" : 1,
"defaultValue" : "0",
"format" : null,
"isSystemField" : false,
"isUniqueId" : false,
"isMultiValue" : false,
"multiValueSeparator" : null,
"fiscalMonthOffset" : 0,
"firstDayOfWeek" : 1,
"canTruncateValue" : true,
"isSkipped" : false,
"decimalSeparator" : ".",
"sortIndex" : 0,
"isSortAscending" : true,
"isComputedField" : true,
"computedFieldExpression" : "if (prev.get('OpportunityId') != curr.get('OpportunityId')) {0} else {curr.get('Amount') - prev.get('Amount')}"
}
First, sorting a CSV is only required if you are adding a computedField to your CSV. Otherwise no sorting is required. Assuming you want to calculate a new field. lets say in the above example you want to calculate the change in opportunity Amount after each update to an opportunity. To do this kind of computation the data must be sorted by 1. OpportunityID 2. SystemModstamp
To achieve this simply set the attribute “sortIndex” : 1 for field OpportunityID and set “sortIndex” : 2 for field SystemModstamp. This would sort the file by OpportunityID and SystemModstamp
- Go to the folder in which you saved the datatset util jar
- Open the file settings.json under config sub folder
- Specify the proxy settings
| Property | Description |
|---|---|
| proxyHost | The host name of the proxy server, if applicable |
| proxyPort | The proxy server port |
| proxyUsername | The username for proxy server authentication |
| proxyPassword | The password for proxy server authentication |
| proxyNtlmDomain | The name of the Windows domain used for NTLM authentication (only when using NTLM proxy) |
Decimal, Currency and Percentage formats are supported in the default local format. The various formats are
| Locale CODE | Currency Format | Decimal Format | Integer Format | Percent Format | Locale Description |
|---|---|---|---|---|---|
| ms_MY | (RM1,234.56) | -1,234.56 | -1,235 | -123,456% | Malay (Malaysia) |
| ar_QA | ر.ق. 1,234.56- | 1,234.56- | 1,235- | -123,456% | Arabic (Qatar) |
| is_IS | -1.235, kr. | -1.234,56 | -1.235 | -123.456% | Icelandic (Iceland) |
| fi_FI | -1 234,56 € | -1 234,56 | -1 235 | -123 456% | Finnish (Finland) |
| en_MT | -€1,234.56 | -1,234.56 | -1,235 | -123,456% | English (Malta) |
| it_CH | SFr.-1’234.56 | -1’234.56 | -1’235 | -123’456% | Italian (Switzerland) |
| nl_BE | -1.234,56 € | -1.234,56 | -1.235 | -123.456% | Dutch (Belgium) |
| ar_SA | ر.س. 1,234.56- | 1,234.56- | 1,235- | -123,456% | Arabic (Saudi Arabia) |
| ar_IQ | د.ع. 1,234.56- | 1,234.56- | 1,235- | -123,456% | Arabic (Iraq) |
| es_PR | ($1,234.56) | -1,234.56 | -1,235 | -123,456% | Spanish (Puerto Rico) |
| es_CL | Ch$-1.235 | -1.234,56 | -1.235 | -123.456% | Spanish (Chile) |
| de_AT | -€ 1.234,56 | -1.234,56 | -1.235 | -123.456% | German (Austria) |
| en_GB | -£1,234.56 | -1,234.56 | -1,235 | -123,456% | English (United Kingdom) |
| es_PA | (B1,234.56) | -1,234.56 | -1,235 | -123,456% | Spanish (Panama) |
| ar_YE | ر.ي. 1,234.56- | 1,234.56- | 1,235- | -123,456% | Arabic (Yemen) |
| mk_MK | -Den 1.234,56 | (1.234,56) | (1.235) | -123.456% | Macedonian (Macedonia) |
| en_CA | -$1,234.56 | -1,234.56 | -1,235 | -123,456% | English (Canada) |
| vi_VN | -1.234,56 đ | -1.234,56 | -1.235 | -123.456% | Vietnamese (Vietnam) |
| nl_NL | € 1.234,56- | -1.234,56 | -1.235 | -123.456% | Dutch (Netherlands) |
| es_US | (US$1,234.56) | -1,234.56 | -1,235 | -123,456% | Spanish (United States) |
| zh_CN | -￥1,234.56 | -1,234.56 | -1,235 | -123,456% | Chinese (China) |
| es_HN | (L1,234.56) | -1,234.56 | -1,235 | -123,456% | Spanish (Honduras) |
| en_US | ($1,234.56) | -1,234.56 | -1,235 | -123,456% | English (United States) |
| ar_MA | د.م. 1,234.56- | 1,234.56- | 1,235- | -123,456% | Arabic (Morocco) |
| in_ID | -Rp1.234,56 | -1.234,56 | -1.235 | -123.456% | Indonesian (Indonesia) |
| en_ZA | R-1,234.56 | -1,234.56 | -1,235 | -123,456% | English (South Africa) |
| ko_KR | -￦1,235 | -1,234.56 | -1,235 | -123,456% | Korean (South Korea) |
| ar_TN | د.ت. 1,234.56- | 1,234.56- | 1,235- | -123,456% | Arabic (Tunisia) |
| sr_RS | -дин. 1.234,56 | -1.234,56 | -1.235 | -123.456% | Serbian (Serbia) |
| be_BY | -Руб1 235 | -1 234,56 | -1 235 | -123 456% | Belarusian (Belarus) |
| zh_TW | -NT$1,234.56 | -1,234.56 | -1,235 | -123,456% | Chinese (Taiwan) |
| ar_SD | ج.س. 1,234.56- | 1,234.56- | 1,235- | -123,456% | Arabic (Sudan) |
| ja_JP_JP_#u-ca-japanese | -￥1,235 | -1,234.56 | -1,235 | -123,456% | Japanese (Japan,JP) |
| es_BO | (B$1.234,56) | -1.234,56 | -1.235 | -123.456% | Spanish (Bolivia) |
| ar_DZ | د.ج. 1,234.56- | 1,234.56- | 1,235- | -123,456% | Arabic (Algeria) |
| es_AR | $-1.234,56 | -1.234,56 | -1.235 | -123.456% | Spanish (Argentina) |
| ar_AE | د.إ. 1,234.56- | 1,234.56- | 1,235- | -123,456% | Arabic (United Arab Emirates) |
| fr_CA | (1 234,56$) | -1 234,56 | -1 235 | -123 456 % | French (Canada) |
| lt_LT | -1 234,56 Lt | -1 234,56 | -1 235 | -12 34 56% | Lithuanian (Lithuania) |
| sr_ME_#Latn | -€ 1,234.56 | -1,234.56 | -1,235 | -123,456% | Serbian (Latin,Montenegro) |
| ar_SY | ل.س. 1,234.56- | 1,234.56- | 1,235- | -123,456% | Arabic (Syria) |
| ru_RU | -1 234,56 руб. | -1 234,56 | -1 235 | -123 456% | Russian (Russia) |
| fr_BE | -1.234,56 € | -1.234,56 | -1.235 | -123.456 % | French (Belgium) |
| es_ES | -1.234,56 € | -1.234,56 | -1.235 | -123.456% | Spanish (Spain) |
| iw_IL | -1,234.56 ש”ח | -1,234.56 | -1,235 | -123,456% | Hebrew (Israel) |
| da_DK | kr -1.234,56 | -1.234,56 | -1.235 | -123.456% | Danish (Denmark) |
| es_CR | (C1,234.56) | -1,234.56 | -1,235 | -123,456% | Spanish (Costa Rica) |
| zh_HK | (HK$1,234.56) | -1,234.56 | -1,235 | -123,456% | Chinese (Hong Kong) |
| ca_ES | -€ 1.234,56 | -1.234,56 | -1.235 | -123.456% | Catalan (Spain) |
| th_TH | ฿-1,234.56 | -1,234.56 | -1,235 | -123,456% | Thai (Thailand) |
| uk_UA | -1 234,56 грн. | -1 234,56 | -1 235 | -123 456% | Ukrainian (Ukraine) |
| es_DO | (RD$1,234.56) | -1,234.56 | -1,235 | -123,456% | Spanish (Dominican Republic) |
| es_VE | Bs.F. -1.234,56 | -1.234,56 | -1.235 | -123.456% | Spanish (Venezuela) |
| pl_PL | -1 234,56 zł | -1 234,56 | -1 235 | -123 456% | Polish (Poland) |
| ar_LY | د.ل. 1,234.56- | 1,234.56- | 1,235- | -123,456% | Arabic (Libya) |
| ar_JO | د.أ. 1,234.56- | 1,234.56- | 1,235- | -123,456% | Arabic (Jordan) |
| hu_HU | -1 234,56 Ft | -1 234,56 | -1 235 | -123 456% | Hungarian (Hungary) |
| es_GT | (Q1,234.56) | -1,234.56 | -1,235 | -123,456% | Spanish (Guatemala) |
| es_PY | (G1.235) | -1.234,56 | -1.235 | -123.456% | Spanish (Paraguay) |
| bg_BG | -лв.1 234,56 | -1 234,56 | -1 235 | -123 456% | Bulgarian (Bulgaria) |
| hr_HR | -Kn 1.234,56 | -1.234,56 | -1.235 | -123.456% | Croatian (Croatia) |
| sr_BA_#Latn | -KM 1,234.56 | -1,234.56 | -1,235 | -123,456% | Serbian (Latin,Bosnia and Herzegovina) |
| ro_RO | -1.234,56 LEI | -1.234,56 | -1.235 | -123.456% | Romanian (Romania) |
| fr_LU | -1 234,56 € | -1 234,56 | -1 235 | -123 456 % | French (Luxembourg) |
| en_SG | -$1,234.56 | -1,234.56 | -1,235 | -123,456% | English (Singapore) |
| es_EC | $-1.234,56 | -1.234,56 | -1.235 | -123.456% | Spanish (Ecuador) |
| sr_BA | -КМ. 1.234,56 | -1.234,56 | -1.235 | -123.456% | Serbian (Bosnia and Herzegovina) |
| es_NI | ($C1,234.56) | -1,234.56 | -1,235 | -123,456% | Spanish (Nicaragua) |
| es_SV | (C1,234.56) | -1,234.56 | -1,235 | -123,456% | Spanish (El Salvador) |
| hi_IN | -रू १,२३४.५६ | -१,२३४.५६ | -१,२३५ | -१२३,४५६% | Hindi (India) |
| el_GR | -1.234,56 € | -1.234,56 | -1.235 | -123.456% | Greek (Greece) |
| sl_SI | -€ 1.234,56 | -1.234,56 | -1.235 | -123.456% | Slovenian (Slovenia) |
| it_IT | -€ 1.234,56 | -1.234,56 | -1.235 | -123.456% | Italian (Italy) |
| ja_JP | -￥1,235 | -1,234.56 | -1,235 | -123,456% | Japanese (Japan) |
| de_LU | -1.234,56 € | -1.234,56 | -1.235 | -123.456% | German (Luxembourg) |
| fr_CH | SFr.-1’234.56 | -1’234.56 | -1’235 | -123’456 % | French (Switzerland) |
| mt_MT | -€1,234.56 | -1,234.56 | -1,235 | -123,456% | Maltese (Malta) |
| ar_BH | د.ب. 1,234.56- | 1,234.56- | 1,235- | -123,456% | Arabic (Bahrain) |
| sr_ME | -€ 1.234,56 | -1.234,56 | -1.235 | -123.456% | Serbian (Montenegro) |
| pt_BR | -R$ 1.234,56 | -1.234,56 | -1.235 | -123.456% | Portuguese (Brazil) |
| no_NO | kr -1 234,56 | -1 234,56 | -1 235 | -123 456% | Norwegian (Norway) |
| de_CH | SFr.-1’234.56 | -1’234.56 | -1’235 | -123’456 % | German (Switzerland) |
| zh_SG | -S$1,234.56 | -1,234.56 | -1,235 | -123,456% | Chinese (Singapore) |
| ar_KW | د.ك. 1,234.56- | 1,234.56- | 1,235- | -123,456% | Arabic (Kuwait) |
| ar_EG | ج.م. 1,234.56- | 1,234.56- | 1,235- | -123,456% | Arabic (Egypt) |
| ga_IE | -€1,234.56 | -1,234.56 | -1,235 | -123,456% | Irish (Ireland) |
| es_PE | S/-1.234,56 | -1.234,56 | -1.235 | -123.456% | Spanish (Peru) |
| cs_CZ | -1 234,56 Kč | -1 234,56 | -1 235 | -123 456% | Czech (Czech Republic) |
| tr_TR | -1.234,56 TL | -1.234,56 | -1.235 | -% 123.456 | Turkish (Turkey) |
| es_UY | (NU$1.234,56) | -1.234,56 | -1.235 | -123.456% | Spanish (Uruguay) |
| en_IE | -€1,234.56 | -1,234.56 | -1,235 | -123,456% | English (Ireland) |
| en_IN | -Rs.1,234.56 | -1,234.56 | -1,235 | -123,456% | English (India) |
| ar_OM | ر.ع. 1,234.56- | 1,234.56- | 1,235- | -123,456% | Arabic (Oman) |
| sr_CS | -CSD 1.234,56 | -1.234,56 | -1.235 | -123.456% | Serbian (Serbia and Montenegro) |
| sq_AL | -Lek1.234,56 | -1.234,56 | -1.235 | -123.456% | Albanian (Albania) |
| pt_PT | -1.234,56 € | -1.234,56 | -1.235 | -123.456% | Portuguese (Portugal) |
| lv_LV | -1 234,56 Ls | -1 234,56 | -1 235 | -123 456% | Latvian (Latvia) |
| sr_RS_#Latn | -din. 1,234.56 | -1,234.56 | -1,235 | -123,456% | Serbian (Latin,Serbia) |
| sk_SK | -1 234,56 € | -1 234,56 | -1 235 | -123 456% | Slovak (Slovakia) |
| es_MX | -$1,234.56 | -1,234.56 | -1,235 | -123,456% | Spanish (Mexico) |
| en_AU | -$1,234.56 | -1,234.56 | -1,235 | -123,456% | English (Australia) |
| no_NO_NY | kr -1 234,56 | -1 234,56 | -1 235 | -123 456% | Norwegian (Norway,Nynorsk) |
| en_NZ | -$1,234.56 | -1,234.56 | -1,235 | -123,456% | English (New Zealand) |
| sv_SE | -1 234,56 kr | -1 234,56 | -1 235 | -123 456% | Swedish (Sweden) |
| ar_LB | ل.ل. 1,234.56- | 1,234.56- | 1,235- | -123,456% | Arabic (Lebanon) |
| de_DE | -1.234,56 € | -1.234,56 | -1.235 | -123.456% | German (Germany) |
| th_TH_TH_#u-nu-thai | ฿-๑,๒๓๔.๕๖ | -๑,๒๓๔.๕๖ | -๑,๒๓๕ | -๑๒๓,๔๕๖% | Thai (Thailand,TH) |
| es_CO | ($1.234,56) | -1.234,56 | -1.235 | -123.456% | Spanish (Colombia) |
| en_PH | (Php1,234.56) | -1,234.56 | -1,235 | -123,456% | English (Philippines) |
| et_EE | -1 234,56 € | -1 234,56 | -1 235 | -123 456% | Estonian (Estonia) |
| el_CY | -€1.234,56 | -1.234,56 | -1.235 | -123.456% | Greek (Cyprus) |
| fr_FR | -1 234,56 € | -1 234,56 | -1 235 | -123 456 % | French (France) |
Warning multiple dataflows will replace the default dataflow on the server. Please create a copy of the default dataflow locally before creating multiple dataflows
The Server only supports a single workflow. This tools enables multiple Dataflows by storing them locally as separate Dataflow files. When the dataflow is executed by the scheduler. This tool will replace the default server workflow with the contents of this local dataflow file and kick off the dataflow on the server. When the dataflow is finished the server dataflow is replaced by a blank dataflow.
In effect the server dataflow is only used as placeholder for the local dataflows. If you want to keep the default server dataflow you must store it locally by creating a new dataflow locally.
First you create a schedule and choose the frequency and the dataflows to run at that schedule. When the schedule is triggered the datasetutil will replace the default server dataflow with the locally scheduled dataflow. when the dataflow finishes then the next dataflow is executed.
- You must login manually through the UI for the scheduler to start
- The datasetutils must be running all the time