# PRD: Merchant Importer Module — Fix 6 Failed Test Cases
## Context
6 QA test cases in the Merchant Importer module are failing. Root causes have been traced to three files:
- `app/Http/Controllers/Merchants/MerchantImportController.php` — missing input validation, broken sample download
- `app/Services/MerchantImportService.php` — duplicate records counted as failures, no special-char normalization
- `public/assets/js/merchant-importer.js` — upload response not guarded against missing `data` key
---
## Fix 1 — MI_008: Upload empty file shows no meaningful error
### Root Cause
`upload()` calls `fgetcsv()` on the opened file handle. When the file is 0 bytes, `fgetcsv()` returns `false`. The code immediately passes that value to `array_map()`, which in PHP 8.x throws a `TypeError: array_map(): Argument #2 must be of type array, bool given`. The exception is caught and the controller returns HTTP 500 via `Helper::rj()`. jQuery's AJAX `error` callback fires and shows only the generic browser `alert('Upload failed. Please check the file and try again.')` — there is no user-readable explanation that the file is empty.
### Expected Behaviour
A clear, specific error message is displayed on the upload area (e.g. toastr or inline message): **"The uploaded file is empty. Please provide a CSV with at least a header row."**
### Changes
**`app/Http/Controllers/Merchants/MerchantImportController.php` — `upload()` method**
```php
$handle = fopen($fullPath, 'r');
$headers = fgetcsv($handle, 0, ',', '"', '');
// ADD immediately after the fgetcsv call:
if ($headers === false || empty(array_filter($headers))) {
fclose($handle);
return response()->json([
'status' => 422,
'message' => 'The uploaded file is empty. Please provide a CSV with at least a header row.',
], 422);
}
```
**`public/assets/js/merchant-importer.js` — `handleFile()` AJAX `error` callback**
```javascript
// REPLACE:
error: function () {
$('#upload-progress').addClass('d-none');
$('#csv-drop-zone').css('opacity', '1');
alert('Upload failed. Please check the file and try again.');
}
// WITH:
error: function (xhr) {
$('#upload-progress').addClass('d-none');
$('#csv-drop-zone').css('opacity', '1');
var msg = 'Upload failed. Please check the file and try again.';
try { msg = JSON.parse(xhr.responseText).message || msg; } catch (e) {}
toastr.error(msg);
}
```
---
## Fix 2 — MI_009: CSV without headers is not rejected
### Root Cause
When a CSV is uploaded without a proper header row (e.g. all cells blank, a file with only data rows, or a file whose first row contains only whitespace), `fgetcsv()` either returns `[null]` (one null element) or an array of empty strings. The controller treats these as valid column names, stores the file, and returns "File uploaded successfully." The mapping table in Step 3 then renders rows with blank column names, and the eventual import silently skips all rows because no column matches any CRM field.
### Expected Behaviour
When the first row of the CSV yields no usable column names, the upload is rejected immediately with a descriptive error: **"CSV file must include a header row as the first line with at least one non-empty column name."**
### Changes
**`app/Http/Controllers/Merchants/MerchantImportController.php` — `upload()` method**
```php
// After the existing empty-file check (Fix 1), ADD:
$headers = array_map('trim', $headers);
$validHeaders = array_filter($headers, fn($h) => $h !== '' && $h !== null);
if (empty($validHeaders)) {
fclose($handle);
return response()->json([
'status' => 422,
'message' => 'CSV file must include a header row as the first line with at least one non-empty column name.',
], 422);
}
// Replace the subsequent array_map with:
$columns = array_map(function ($header) use ($sample) {
return ['name' => $header, 'sample' => $sample[$header] ?? ''];
}, $headers); // uses the already-trimmed $headers
```
Also strip the UTF-8 BOM that Windows-generated CSVs prepend to the first header, which causes silent mapping failures:
```php
// ADD before the empty-check:
$headers = fgetcsv($handle, 0, ',', '"', '');
// Strip UTF-8 BOM from first header if present
if (!empty($headers[0])) {
$headers[0] = ltrim($headers[0], "\xEF\xBB\xBF");
}
```
---
## Fix 3 — MI_010: Sample CSV download produces an empty file
### Root Cause
`downloadSample()` uses `fopen('php://output', 'w')` and then calls `ob_start()`. In PHP 8.x under Laravel's own output-buffering stack, writes to `php://output` bypass the `ob_start()` buffer and go directly to the SAPI output layer. Consequently `ob_get_clean()` returns an empty string, and `return response($content, 200, [...])` sends a 0-byte file to the browser.
### Expected Behaviour
Clicking **Download Sample CSV** downloads a valid CSV file (`merchant_import_sample.csv`) containing one header row and one sample data row.
### Changes
**`app/Http/Controllers/Merchants/MerchantImportController.php` — `downloadSample()` method**
```php
// REPLACE the entire method body:
public function downloadSample()
{
$headers = ['mid', 'dba', 'legal_name', 'phone', 'processor', 'datasource', 'approval_date', 'note'];
$sample = ['MID123456', 'Sample DBA Name', 'Sample Legal LLC', '555-123-4567', 'First Data', 'Manual', '2024-01-15', 'Sample notes here'];
$stream = fopen('php://temp', 'r+w');
fputcsv($stream, $headers, ',', '"', '');
fputcsv($stream, $sample, ',', '"', '');
rewind($stream);
$content = stream_get_contents($stream);
fclose($stream);
return response($content, 200, [
'Content-Type' => 'text/csv',
'Content-Disposition' => 'attachment; filename="merchant_import_sample.csv"',
]);
}
```
---
## Fix 4 — MI_013: Uploading a valid file does not advance to the Select Defaults step
### Root Cause
Two issues prevent the wizard from advancing to Step 2 after a successful upload:
**Issue A — Response guard missing in JS:** The AJAX success callback accesses `res.data.path` and `res.data.columns` without first confirming that `res.data` exists. If the server response wraps the payload differently (e.g. a validation exception that returns HTTP 200 with an error body), a `TypeError: Cannot read properties of undefined (reading 'path')` is thrown silently inside the callback. The `unlockAndGoTo(2)` call that follows is never reached and the wizard stays on Step 1 with no visible feedback.
**Issue B — Select2 not re-initialised after pane becomes visible:** `initSelect2($('#importer-pane-2'))` is called once at page load when the pane has `d-none`. Some Select2 versions fail to compute `width: '100%'` on hidden elements, so the dropdowns in Step 2 render with zero width and appear invisible when the pane is later shown.
### Expected Behaviour
After a valid CSV is successfully uploaded, the wizard advances to the **Select Defaults** step (pane 2), and all three dropdowns (Processor, Group, Merchant Type) render at full width.
### Changes
**`public/assets/js/merchant-importer.js` — `handleFile()` success callback**
```javascript
// REPLACE:
success: function (res) {
$('#upload-progress').addClass('d-none');
$('#csv-drop-zone').css('opacity', '1');
if (res && res.status === 200) {
state.filePath = res.data.path;
...
} else {
alert(res.message || 'Upload failed. Please try again.');
}
},
// WITH:
success: function (res) {
$('#upload-progress').addClass('d-none');
$('#csv-drop-zone').css('opacity', '1');
if (res && res.status === 200 && res.data) {
state.filePath = res.data.path || null;
state.originalName = res.data.original_name || null;
state.columns = res.data.columns || [];
buildMappingTable();
unlockAndGoTo(2);
// Re-initialise Select2 after pane-2 becomes visible
setTimeout(function () { initSelect2($('#importer-pane-2')); }, 50);
} else {
toastr.error((res && res.message) ? res.message : 'Upload failed. Please try again.');
}
},
```
---
## Fix 5 — MI_021: Special characters in CSV are not handled correctly
### Root Cause
The controller reads each CSV row with `fgetcsv()` which correctly handles quoted fields, but passes raw values directly into `Merchant::create($input)` without any sanitisation. Two failure modes arise:
1. **Multi-byte / non-ASCII characters (accents, symbols, Unicode):** If the CSV is not UTF-8 encoded (e.g. Windows-1252 / Latin-1), the data is stored as garbled bytes. MySQL with `utf8mb4` will silently truncate or raise a "Incorrect string value" error, which is caught per-row, and the row is counted as `$failed` with an opaque DB error message.
2. **Leading/trailing control characters and non-printable bytes:** Fields like `mid` may arrive with invisible chars (BOM remnants, non-breaking spaces) that cause the MID uniqueness check to wrongly pass, and the subsequent `Merchant::create()` to insert a near-duplicate.
### Expected Behaviour
- CSV files encoded in UTF-8 or UTF-8 with BOM are imported without character corruption.
- Leading/trailing whitespace and non-printable characters are stripped from every field value before insertion.
- If the encoding is not UTF-8-compatible, an import warning is logged (not a hard failure), and the field is stored with replacement characters.
### Changes
**`app/Services/MerchantImportService.php` — inside the `while` data-row loop**
```php
// REPLACE:
$input[$crmField] = trim($csvRow[$csvCol]);
// WITH:
$raw = trim($csvRow[$csvCol]);
// Convert non-UTF-8 input to UTF-8; replace unmappable chars with '?'
if (!mb_check_encoding($raw, 'UTF-8')) {
$raw = mb_convert_encoding($raw, 'UTF-8', 'Windows-1252');
}
// Strip non-printable / control characters (except CR/LF/TAB)
$raw = preg_replace('/[\x00-\x08\x0B\x0C\x0E-\x1F\x7F]/u', '', $raw);
$input[$crmField] = $raw;
```
**`app/Services/MerchantImportService.php` — inside `process()`, strip BOM from first column of first data row (defensive)**
```php
// ADD directly after array_combine($csvHeaders, $row):
$csvRow = array_combine($csvHeaders, $row);
// Strip BOM artifact from first field value
$firstKey = array_key_first($csvRow);
if ($firstKey !== null) {
$csvRow[$firstKey] = ltrim($csvRow[$firstKey], "\xEF\xBB\xBF");
}
```
---
## Fix 6 — MI_022: Duplicate records are logged as failures instead of being handled per defined logic
### Root Cause
When the MID of a CSV row already exists in the database for the same company, the service adds the row to `$errors[]` and increments `$failed`. This means:
1. Duplicates appear alongside hard data-errors (missing MID, column mismatch) in the error table — there is no distinction between a "known duplicate" and a "broken row".
2. The import is marked `status = 'failed'` if ALL rows are duplicates, even though no actual processing error occurred.
3. There is no way for the user to choose an action (Skip / Update) for duplicate MIDs, unlike the Lead Importer which offers three options.
### Expected Behaviour
- Duplicate MIDs are counted separately as **skipped** (not failed).
- A new **Duplicate Action** dropdown is added to the **Select Defaults** step (Step 2): **Skip** (default) or **Update existing merchant**.
- The import result summary shows a fourth stat card: **Skipped (Duplicates)**.
- When action = **Update**, the service updates the existing merchant record's non-MID fields instead of inserting.
### Changes
**`resources/views/admin/merchants/merchant-importer.blade.php` — Step 2 defaults pane**
```html