Data Processing Tool FAQ

Bulk Data Processing Speed

• FAQ: Is bulk data processing faster than the standard data import? Can a dedicated database further improve speed?
• Answer: Yes. Bulk data processing is typically faster than the standard data import. In public cloud environments there are speed limits for bulk processing, but you can improve throughput by configuring a dedicated database and manually adjusting thresholds. Standard data import processes about 50 records per batch in roughly one second, while bulk processing is faster.

Import File Size Limits

• FAQ: What is the maximum file size supported by the standard import and bulk processing features?
• Answer: The standard import limits vary depending on workflows and approval flows:
  • No workflows or approval flows triggered: up to 300,000 records, maximum 50 MB.
  • Workflows triggered but not approval flows: up to 1,000 records, maximum 2 MB.
  • Both workflows and approval flows triggered: up to 500 records, maximum 2 MB.
• The bulk processing feature supports a maximum file size of 100 MB.

Feature Permission Control

• FAQ: Can bulk data processing use feature permissions to control the Import, Export, and Delete buttons? Can backend objects be included in the permission model to differentiate Roles and assign button permissions?
• Answer: Currently bulk data processing does not support controlling buttons through feature permissions. This capability is planned; in a future release you will be able to assign button permissions based on Roles.

Integration with File Servers

• FAQ: Can bulk data processing integrate with file servers—for example, a salesperson uploads files to a specific location and a scheduled job reads and parses them?
• Answer: The current import file feature does not support direct integration with file servers. If sales staff need to upload files to a Cloud Drive on a schedule, evaluate whether this approach and its error-handling procedures are appropriate and necessary. Currently, the export function supports integration with Alibaba Cloud or self-hosted FTP services.

Import/Export Speed Optimization

• FAQ: Can the speed of standard import/export be optimized? Even after optimization, will there still be a significant gap compared to bulk processing?
• Answer: In the short term, the standard import/export speed cannot be significantly optimized. Even with improvements, when workflows and approval flows are triggered the system must perform complex validations, so speed gains are limited. By contrast, bulk processing does not trigger those flows and therefore maintains a performance advantage.