-
Notifications
You must be signed in to change notification settings - Fork 64
The easiest solution is to use JRE 8 or higher you will not have these SSL Handshake errors.
The version of the JCE policy files that are bundled with JRE 7 environment allow limited cryptography to be used. To uses TLSV1.2 you must download the “unlimited strength” policy files which contain no restrictions on cryptographic strengths. Refer to the download readme for installation instructions.
If you get the Error Message “Logon Failed- LOGIN_MUST_USE_SECURITY_TOKEN: Inv… ” when attempting to login using datasetutils. The Cause, is that the machine you are running datasetutils on is not part of is not part of the SalesForce Trusted IP Ranges. There are 2 ways to resolve this error.
Resolution 1:
- Log into SalesForce with an Administration account
- Go to “Account Setup” by clicking the username in the top left panel and selecting “Setup”
- Select “Security Controls” > “Network Access” from the menu on the right.
- Click “New” to add a new trusted IP range and enter the range of External IP addresses that the datsetutils will use when accessing salesforce.
Resolution 2:
- To receive a new security token, sign in to Salesforce org
- In Salesforce, click your username at the top, and then select Setup.
- Under My Personal Information, click Reset your security token. You will get your token in the email associated with the salesforce login.
- When connecting to Salesforce. In the password field enter password+token, for example is password is ‘1234’ and token ‘randomtokenstring’. then enter ‘1234randomtokenstring’
If some rows fail because of data issues then that row is skipped and written to an error file <file_name>_err.csv, This file has the error and the original row that failed. The rest of the file is uploaded and processed.
If the number of failures exceeds 10,000 then the entire file is rejected and processing is stopped
If you want to upsert data to a dataset then that dataset must have a UniqueId defined. You can do that by editing the json and setting the attribute ‘isUniqueId=true’ for that field
Only fields of type Text can be set as UniqueId. Only one field can be set as UniqueId The schema of the target dataset must exactly match the schema of the file being loaded.
{
"name" : "MyId",
"fullyQualifiedName" : "MyId",
"label" : "MyId",
"description" : "MyId",
"type" : "Text",
"precision" : 18,
"scale" : 0,
"defaultValue" : "0",
"format" : null,
"isSystemField" : false,
"isUniqueId" : true,
"isMultiValue" : false,
"multiValueSeparator" : null,
"fiscalMonthOffset" : 0,
"firstDayOfWeek" : 1,
"canTruncateValue" : true,
"isSkipped" : false,
"decimalSeparator" : "."
}
- First run the upload action for your csv. This will generate the default json for your csv
- Then Edit the json (use http://www.jsoneditoronline.org/) and at the end of the last field add a new field.
- Make sure the type of the field is the return type of computation formula
- In the field set attribute ‘isComputedField’ to true
- In the field set attribute ‘computedFieldExpression’ to the formula you want to use to calculate your field, for example:
`"computedFieldExpression" : "if (prev.get('OpportunityId') != curr.get('OpportunityId')) {0} else {curr.get('Amount') - prev.get('Amount')}"`
The syntax for this Expression is Java Script. In theory you can do any calculations that you can do in standard java script engine
Complete Field definition from above example:
{
"name" : "AmountDelta",
"fullyQualifiedName" : "AmountDelta",
"label" : "AmountDelta",
"description" : "AmountDelta",
"type" : "Numeric",
"precision" : 18,
"scale" : 1,
"defaultValue" : "0",
"format" : null,
"isSystemField" : false,
"isUniqueId" : false,
"isMultiValue" : false,
"multiValueSeparator" : null,
"fiscalMonthOffset" : 0,
"firstDayOfWeek" : 1,
"canTruncateValue" : true,
"isSkipped" : false,
"decimalSeparator" : ".",
"sortIndex" : 0,
"isSortAscending" : true,
"isComputedField" : true,
"computedFieldExpression" : "if (prev.get('OpportunityId') != curr.get('OpportunityId')) {0} else {curr.get('Amount') - prev.get('Amount')}"
}
First, sorting a CSV is only required if you are adding a computedField to your CSV. Otherwise no sorting is required. Assuming you want to calculate a new field. lets say in the above example you want to calculate the change in opportunity Amount after each update to an opportunity. To do this kind of computation the data must be sorted by 1. OpportunityID 2. SystemModstamp
To achieve this simply set the attribute “sortIndex” : 1 for field OpportunityID and set “sortIndex” : 2 for field SystemModstamp. This would sort the file by OpportunityID and SystemModstamp
- Click on Menu item in top right corner of the login screen
- Click on Settings Menu
- Specify the proxy settings
| Property | Description |
|---|---|
| proxyHost | The host name of the proxy server, if applicable |
| proxyPort | The proxy server port |
| proxyUsername | The username for proxy server authentication |
| proxyPassword | The password for proxy server authentication |
| proxyNtlmDomain | The name of the Windows domain used for NTLM authentication (only when using NTLM proxy) |
Warning multiple dataflows will replace the default dataflow on the server. Please create a copy of the default dataflow locally before creating multiple dataflows
The Server only supports a single workflow. This tools enables multiple Dataflows by storing them locally as separate Dataflow files. When the dataflow is executed by the local scheduler. This tool will replace the default server workflow with the contents of this local dataflow file and kick off the dataflow on the server. When the dataflow is finished the server dataflow is replaced by a blank dataflow. In effect the server dataflow is only used as placeholder for the local dataflows. If you want to keep the default server dataflow you must store it locally by creating a new dataflow locally.
First you create a schedule and choose the frequency at which the dataflow should be executed. When the schedule is triggered the datasetutil will replace the default server dataflow with the locally scheduled dataflow. when the dataflow finishes then the next dataflow is executed.
Once you save the schedule, you must enable it.
If you want to edit the schedule you must disable it first.
- You must login manually through the UI for the scheduler to start
- The datasetutils must be running all the time
- If you choose to run multiple dataflows in the same schedule. The flwos will be executed in the same order you select them in the UI.
File listeners watch for file to appear in the directory. Once a file appears the server automatically picks it up (along with metadata json if present) and uploads it to the dataset configured in the listener
- You must login manually through the UI for the listener to start
- The datasetutils must be running all the time
- The listener directory cannot be a top level directory.
- As a best practice you should create separate directories for different files and datasets
- The JSON file must be present in the same directory and should have the name <filename>_schema.json. If JSON file is not found the server will generate a new one every time
- You must enable a listener after you save it.
- You must disable a listener before you can edit it.
- After a file is processed it is moved to success or error directories in the parent of the listener directory depending on the status of the upload.
Decimal, Currency and Percentage formats are supported in the default local format. The various formats are
| Locale CODE | Currency Format | Decimal Format | Integer Format | Percent Format | Locale Description |
|---|---|---|---|---|---|
| ms_MY | (RM1,234.56) | -1,234.56 | -1,235 | -123,456% | Malay (Malaysia) |
| ar_QA | ر.ق. 1,234.56- | 1,234.56- | 1,235- | -123,456% | Arabic (Qatar) |
| is_IS | -1.235, kr. | -1.234,56 | -1.235 | -123.456% | Icelandic (Iceland) |
| fi_FI | -1 234,56 € | -1 234,56 | -1 235 | -123 456% | Finnish (Finland) |
| en_MT | -€1,234.56 | -1,234.56 | -1,235 | -123,456% | English (Malta) |
| it_CH | SFr.-1’234.56 | -1’234.56 | -1’235 | -123’456% | Italian (Switzerland) |
| nl_BE | -1.234,56 € | -1.234,56 | -1.235 | -123.456% | Dutch (Belgium) |
| ar_SA | ر.س. 1,234.56- | 1,234.56- | 1,235- | -123,456% | Arabic (Saudi Arabia) |
| ar_IQ | د.ع. 1,234.56- | 1,234.56- | 1,235- | -123,456% | Arabic (Iraq) |
| es_PR | ($1,234.56) | -1,234.56 | -1,235 | -123,456% | Spanish (Puerto Rico) |
| es_CL | Ch$-1.235 | -1.234,56 | -1.235 | -123.456% | Spanish (Chile) |
| de_AT | -€ 1.234,56 | -1.234,56 | -1.235 | -123.456% | German (Austria) |
| en_GB | -£1,234.56 | -1,234.56 | -1,235 | -123,456% | English (United Kingdom) |
| es_PA | (B1,234.56) | -1,234.56 | -1,235 | -123,456% | Spanish (Panama) |
| ar_YE | ر.ي. 1,234.56- | 1,234.56- | 1,235- | -123,456% | Arabic (Yemen) |
| mk_MK | -Den 1.234,56 | (1.234,56) | (1.235) | -123.456% | Macedonian (Macedonia) |
| en_CA | -$1,234.56 | -1,234.56 | -1,235 | -123,456% | English (Canada) |
| vi_VN | -1.234,56 đ | -1.234,56 | -1.235 | -123.456% | Vietnamese (Vietnam) |
| nl_NL | € 1.234,56- | -1.234,56 | -1.235 | -123.456% | Dutch (Netherlands) |
| es_US | (US$1,234.56) | -1,234.56 | -1,235 | -123,456% | Spanish (United States) |
| zh_CN | -￥1,234.56 | -1,234.56 | -1,235 | -123,456% | Chinese (China) |
| es_HN | (L1,234.56) | -1,234.56 | -1,235 | -123,456% | Spanish (Honduras) |
| en_US | ($1,234.56) | -1,234.56 | -1,235 | -123,456% | English (United States) |
| ar_MA | د.م. 1,234.56- | 1,234.56- | 1,235- | -123,456% | Arabic (Morocco) |
| in_ID | -Rp1.234,56 | -1.234,56 | -1.235 | -123.456% | Indonesian (Indonesia) |
| en_ZA | R-1,234.56 | -1,234.56 | -1,235 | -123,456% | English (South Africa) |
| ko_KR | -￦1,235 | -1,234.56 | -1,235 | -123,456% | Korean (South Korea) |
| ar_TN | د.ت. 1,234.56- | 1,234.56- | 1,235- | -123,456% | Arabic (Tunisia) |
| sr_RS | -дин. 1.234,56 | -1.234,56 | -1.235 | -123.456% | Serbian (Serbia) |
| be_BY | -Руб1 235 | -1 234,56 | -1 235 | -123 456% | Belarusian (Belarus) |
| zh_TW | -NT$1,234.56 | -1,234.56 | -1,235 | -123,456% | Chinese (Taiwan) |
| ar_SD | ج.س. 1,234.56- | 1,234.56- | 1,235- | -123,456% | Arabic (Sudan) |
| ja_JP_JP_#u-ca-japanese | -￥1,235 | -1,234.56 | -1,235 | -123,456% | Japanese (Japan,JP) |
| es_BO | (B$1.234,56) | -1.234,56 | -1.235 | -123.456% | Spanish (Bolivia) |
| ar_DZ | د.ج. 1,234.56- | 1,234.56- | 1,235- | -123,456% | Arabic (Algeria) |
| es_AR | $-1.234,56 | -1.234,56 | -1.235 | -123.456% | Spanish (Argentina) |
| ar_AE | د.إ. 1,234.56- | 1,234.56- | 1,235- | -123,456% | Arabic (United Arab Emirates) |
| fr_CA | (1 234,56$) | -1 234,56 | -1 235 | -123 456 % | French (Canada) |
| lt_LT | -1 234,56 Lt | -1 234,56 | -1 235 | -12 34 56% | Lithuanian (Lithuania) |
| sr_ME_#Latn | -€ 1,234.56 | -1,234.56 | -1,235 | -123,456% | Serbian (Latin,Montenegro) |
| ar_SY | ل.س. 1,234.56- | 1,234.56- | 1,235- | -123,456% | Arabic (Syria) |
| ru_RU | -1 234,56 руб. | -1 234,56 | -1 235 | -123 456% | Russian (Russia) |
| fr_BE | -1.234,56 € | -1.234,56 | -1.235 | -123.456 % | French (Belgium) |
| es_ES | -1.234,56 € | -1.234,56 | -1.235 | -123.456% | Spanish (Spain) |
| iw_IL | -1,234.56 ש”ח | -1,234.56 | -1,235 | -123,456% | Hebrew (Israel) |
| da_DK | kr -1.234,56 | -1.234,56 | -1.235 | -123.456% | Danish (Denmark) |
| es_CR | (C1,234.56) | -1,234.56 | -1,235 | -123,456% | Spanish (Costa Rica) |
| zh_HK | (HK$1,234.56) | -1,234.56 | -1,235 | -123,456% | Chinese (Hong Kong) |
| ca_ES | -€ 1.234,56 | -1.234,56 | -1.235 | -123.456% | Catalan (Spain) |
| th_TH | ฿-1,234.56 | -1,234.56 | -1,235 | -123,456% | Thai (Thailand) |
| uk_UA | -1 234,56 грн. | -1 234,56 | -1 235 | -123 456% | Ukrainian (Ukraine) |
| es_DO | (RD$1,234.56) | -1,234.56 | -1,235 | -123,456% | Spanish (Dominican Republic) |
| es_VE | Bs.F. -1.234,56 | -1.234,56 | -1.235 | -123.456% | Spanish (Venezuela) |
| pl_PL | -1 234,56 zł | -1 234,56 | -1 235 | -123 456% | Polish (Poland) |
| ar_LY | د.ل. 1,234.56- | 1,234.56- | 1,235- | -123,456% | Arabic (Libya) |
| ar_JO | د.أ. 1,234.56- | 1,234.56- | 1,235- | -123,456% | Arabic (Jordan) |
| hu_HU | -1 234,56 Ft | -1 234,56 | -1 235 | -123 456% | Hungarian (Hungary) |
| es_GT | (Q1,234.56) | -1,234.56 | -1,235 | -123,456% | Spanish (Guatemala) |
| es_PY | (G1.235) | -1.234,56 | -1.235 | -123.456% | Spanish (Paraguay) |
| bg_BG | -лв.1 234,56 | -1 234,56 | -1 235 | -123 456% | Bulgarian (Bulgaria) |
| hr_HR | -Kn 1.234,56 | -1.234,56 | -1.235 | -123.456% | Croatian (Croatia) |
| sr_BA_#Latn | -KM 1,234.56 | -1,234.56 | -1,235 | -123,456% | Serbian (Latin,Bosnia and Herzegovina) |
| ro_RO | -1.234,56 LEI | -1.234,56 | -1.235 | -123.456% | Romanian (Romania) |
| fr_LU | -1 234,56 € | -1 234,56 | -1 235 | -123 456 % | French (Luxembourg) |
| en_SG | -$1,234.56 | -1,234.56 | -1,235 | -123,456% | English (Singapore) |
| es_EC | $-1.234,56 | -1.234,56 | -1.235 | -123.456% | Spanish (Ecuador) |
| sr_BA | -КМ. 1.234,56 | -1.234,56 | -1.235 | -123.456% | Serbian (Bosnia and Herzegovina) |
| es_NI | ($C1,234.56) | -1,234.56 | -1,235 | -123,456% | Spanish (Nicaragua) |
| es_SV | (C1,234.56) | -1,234.56 | -1,235 | -123,456% | Spanish (El Salvador) |
| hi_IN | -रू १,२३४.५६ | -१,२३४.५६ | -१,२३५ | -१२३,४५६% | Hindi (India) |
| el_GR | -1.234,56 € | -1.234,56 | -1.235 | -123.456% | Greek (Greece) |
| sl_SI | -€ 1.234,56 | -1.234,56 | -1.235 | -123.456% | Slovenian (Slovenia) |
| it_IT | -€ 1.234,56 | -1.234,56 | -1.235 | -123.456% | Italian (Italy) |
| ja_JP | -￥1,235 | -1,234.56 | -1,235 | -123,456% | Japanese (Japan) |
| de_LU | -1.234,56 € | -1.234,56 | -1.235 | -123.456% | German (Luxembourg) |
| fr_CH | SFr.-1’234.56 | -1’234.56 | -1’235 | -123’456 % | French (Switzerland) |
| mt_MT | -€1,234.56 | -1,234.56 | -1,235 | -123,456% | Maltese (Malta) |
| ar_BH | د.ب. 1,234.56- | 1,234.56- | 1,235- | -123,456% | Arabic (Bahrain) |
| sr_ME | -€ 1.234,56 | -1.234,56 | -1.235 | -123.456% | Serbian (Montenegro) |
| pt_BR | -R$ 1.234,56 | -1.234,56 | -1.235 | -123.456% | Portuguese (Brazil) |
| no_NO | kr -1 234,56 | -1 234,56 | -1 235 | -123 456% | Norwegian (Norway) |
| de_CH | SFr.-1’234.56 | -1’234.56 | -1’235 | -123’456 % | German (Switzerland) |
| zh_SG | -S$1,234.56 | -1,234.56 | -1,235 | -123,456% | Chinese (Singapore) |
| ar_KW | د.ك. 1,234.56- | 1,234.56- | 1,235- | -123,456% | Arabic (Kuwait) |
| ar_EG | ج.م. 1,234.56- | 1,234.56- | 1,235- | -123,456% | Arabic (Egypt) |
| ga_IE | -€1,234.56 | -1,234.56 | -1,235 | -123,456% | Irish (Ireland) |
| es_PE | S/-1.234,56 | -1.234,56 | -1.235 | -123.456% | Spanish (Peru) |
| cs_CZ | -1 234,56 Kč | -1 234,56 | -1 235 | -123 456% | Czech (Czech Republic) |
| tr_TR | -1.234,56 TL | -1.234,56 | -1.235 | -% 123.456 | Turkish (Turkey) |
| es_UY | (NU$1.234,56) | -1.234,56 | -1.235 | -123.456% | Spanish (Uruguay) |
| en_IE | -€1,234.56 | -1,234.56 | -1,235 | -123,456% | English (Ireland) |
| en_IN | -Rs.1,234.56 | -1,234.56 | -1,235 | -123,456% | English (India) |
| ar_OM | ر.ع. 1,234.56- | 1,234.56- | 1,235- | -123,456% | Arabic (Oman) |
| sr_CS | -CSD 1.234,56 | -1.234,56 | -1.235 | -123.456% | Serbian (Serbia and Montenegro) |
| sq_AL | -Lek1.234,56 | -1.234,56 | -1.235 | -123.456% | Albanian (Albania) |
| pt_PT | -1.234,56 € | -1.234,56 | -1.235 | -123.456% | Portuguese (Portugal) |
| lv_LV | -1 234,56 Ls | -1 234,56 | -1 235 | -123 456% | Latvian (Latvia) |
| sr_RS_#Latn | -din. 1,234.56 | -1,234.56 | -1,235 | -123,456% | Serbian (Latin,Serbia) |
| sk_SK | -1 234,56 € | -1 234,56 | -1 235 | -123 456% | Slovak (Slovakia) |
| es_MX | -$1,234.56 | -1,234.56 | -1,235 | -123,456% | Spanish (Mexico) |
| en_AU | -$1,234.56 | -1,234.56 | -1,235 | -123,456% | English (Australia) |
| no_NO_NY | kr -1 234,56 | -1 234,56 | -1 235 | -123 456% | Norwegian (Norway,Nynorsk) |
| en_NZ | -$1,234.56 | -1,234.56 | -1,235 | -123,456% | English (New Zealand) |
| sv_SE | -1 234,56 kr | -1 234,56 | -1 235 | -123 456% | Swedish (Sweden) |
| ar_LB | ل.ل. 1,234.56- | 1,234.56- | 1,235- | -123,456% | Arabic (Lebanon) |
| de_DE | -1.234,56 € | -1.234,56 | -1.235 | -123.456% | German (Germany) |
| th_TH_TH_#u-nu-thai | ฿-๑,๒๓๔.๕๖ | -๑,๒๓๔.๕๖ | -๑,๒๓๕ | -๑๒๓,๔๕๖% | Thai (Thailand,TH) |
| es_CO | ($1.234,56) | -1.234,56 | -1.235 | -123.456% | Spanish (Colombia) |
| en_PH | (Php1,234.56) | -1,234.56 | -1,235 | -123,456% | English (Philippines) |
| et_EE | -1 234,56 € | -1 234,56 | -1 235 | -123 456% | Estonian (Estonia) |
| el_CY | -€1.234,56 | -1.234,56 | -1.235 | -123.456% | Greek (Cyprus) |
| fr_FR | -1 234,56 € | -1 234,56 | -1 235 | -123 456 % | French (France) |