Recomputes colHeaders from headerColumns/cols for the current
useLabels state, and pushes it into a live grid via updateSettings
if one is mounted (no refetch needed).
Shows the DC data columns present in the latest VA message and hides the
rest (primary-key and validator-hidden columns are always preserved). VA
parameters can add/remove columns between pushes.
Stop the bulk-validation flow (paste / autofill): invalidate in-flight
responses via the epoch counter, reset spinner cells, hide the banner,
and (when triggered from the banner's Cancel button) undo the change.
This function takes row and column numbers for the cel to be validated and pouplated with values.
It will send the row values without the current column to the sas.
Sas will return values and if length greater then zero cel becomes dropdown type and values from sas
put to source.
Reads the current multi-column sort config defensively. The sorting plugin
may not be initialised yet (e.g. editTable() invoked right after load in VA
embed mode, before the grid finished setup), in which case getSortConfig()
throws on null internal state. There's no sort to preserve then, so [].
Parses values of extended cell validation for the given dynamic cell validation entry
we need to get cellValue from that row and not from hashed cell validation source.
if not provided, row that is used is row found in cellValidationEntry. This is needed when for example we change the w in hot
we need to get cellValue from that row and not from hashed cell validation source.
BASE_COL names of every HARDFORMULA/SOFTFORMULA rule whose result is
expected to be stable for a given row (see getStableFormulaBaseCols
for why DC.USER_NAME/DC.ORIG_VALUE/DC.ROW_STATUS-based rules are
excluded). Used only by overlayFormulaRawValuesOnUnchanged and
cancelEdit's preserve-on-cancel patch, which exist specifically to work
around dataSourceUnchanged's formula-only raw overlay - see those
methods' own doc comments for why that stays narrower than
getRevertableColumnNames(). NOT the same filter seedFormulaValuesForRow
uses above, which seeds every formula column regardless of this
distinction.
Every column eligible to be checked/marked as "overwritten" - every
revertable column (see getRevertableCols), not just the narrower
formula-only set getFormulaBaseCols() returns. Used by
markOverwrittenCells and the live afterChange sync - NOT by
overlayFormulaRawValuesOnUnchanged/cancelEdit's preserve-on-cancel
patch, which stay scoped to getFormulaBaseCols() specifically (see
those methods' own doc comments for why).
Puts the page into its getdata error state. Shared by the callService
.catch and initSetup's own retry branch: a throw from the retry runs on a
bare setTimeout, outside that promise chain, so it would otherwise never
reach the .catch and the user would be left on the loading spinner.
Handles one VA message immediately (NOT debounced): updates column
visibility and decides the filter. The cheap work (visibility, building the
filter clauses, deciding pending) runs per message so the "pending"
indicator can show the instant VA posts; only the expensive APPLY (saveQuery
reload) is debounced (live mode) via scheduleVaFilterApply.
Filter data always comes from DC's getdata; VA only drives (a) column
visibility and (b) the active filter (its params -> DC clauses, applied by
reloading at /editor/
A derived (computed) column - configured in MPE_DERIVED, created at read
time. Read-only, and styled apart from an ordinary read-only column so the
user can see that there is nothing to edit rather than that they may not.
Marks every cell whose current value differs from what SAS actually
sent for it (see findOverwrittenCells) with a read-only comment
showing the original value - a HARDFORMULA/SOFTFORMULA rule silently
overwriting real pre-existing data is one way this happens, but so is
any direct edit; both need the same "here's what it used to be, and a
way back" treatment. Must run after hot.updateSettings() has processed
the formulas: getDataAtRowProp only resolves a formula cell's live
computed value once HyperFormula has actually evaluated it, not the
raw formula string dataSource holds right after applyFormulaRules.
Also flips each affected row's EDIT_STATUS to 'M' - this is a real,
permanent difference from the raw dataset (every future load
recomputes the same formula again, and a direct edit made before this
method ever ran isn't something afterChange would have seen), so
nothing else would otherwise mark these rows modified. Doing this once
here, before dataSourceUnchanged is ever cloned from dataSource, means
the 'M' baseline is already shared by both snapshots - no special
handling needed to keep it from being wiped out by a later
cancelEdit().
VA mode, initial (unfiltered) route only: briefly wait for VA's buffered
first message and, if it already carries a filter, save that filter and
navigate to /editor/
/ so the very first getdata is
filtered. Returns true if it navigated (caller must abort its own load),
false to proceed with the normal unfiltered load.
The filter is built directly from VA parameters (param label === DC column,
which VA↔DC matching already requires; numeric inferred from the VA param
dataType/value) because DC column metadata isn't available until getdata
has run. Any failure falls back to the normal load; the post-load replay
(with full metadata) reconciles to the same filter signature.
Overlays the true raw (pre-formula) value onto dataSourceUnchanged for
every HARDFORMULA/SOFTFORMULA base col that already had real data (see
markOverwrittenCells) - mutates in place. A HARDFORMULA/SOFTFORMULA
rule can silently overwrite a value that already existed in the real
dataset - that's a real change classifyRow's diff should pick up, not
something that only becomes visible once an edit session starts. Shared
by the initial load (so the row header agrees with EDIT_STATUS from the
very first render) and editTable() (every time the baseline is rebuilt).
After any change or update to the hot datasource we lose cell validation values.
This function is called in those places, to update all cells with values if existing.
Note that was discussed:
Rows with same data does not have arrows until you click on them (arrows gets lost after addRow)
That is because this function resets the values for the found hashes, and if multiple rows have same data,
hash table contains only first row that is hashed, so others don't get arrow re-set
when we don't want to re-set force values of every row in the table
A primary key identifies its own row for the rest of the edit session
(getRowKey, dataModified, classifyRow all match rows by comparing PK
values) - a HARDFORMULA/SOFTFORMULA rule on a PK column gets seeded by
applyFormulaRules as a live =... formula string exactly like any
other formula column, but a PK can never be allowed to keep
recalculating for the rest of the session the way a normal formula
column does: saveTable's own formulaRules loop only resolves a formula
column's computed value once, at submit, so anything that changed a
column the PK's formula depends on in between would silently shift the
PK out from under dataModified/classifyRow's PK-matching - the same
failure mode the afterChange hook below already prevents for a
user-typed PK formula. So a PK-as-formula-column is frozen to its
computed value once, right here at load. Both dataSource and
dataSourceUnchanged get the same frozen value - overriding whatever
overlayFormulaRawValuesOnUnchanged set for it, since a PK's identity
role takes priority over the normal "did a formula overwrite real
data" modified-detection every other formula column gets.
Drive dynamic-source load + validation for cells that were bulk-filled
(paste or autofill). HARDSELECT_HOOK / SOFTSELECT_HOOK columns need a SAS
roundtrip; non-hook cells just need HOT's static validators. Caps backend
concurrency, uses an epoch so a cancelled run can't mutate later state,
and gates >3-cell runs behind a confirm modal.
Measures whether the applied-filter clause fits the collapsed single line,
and shows or hides the chevron accordingly. The measurement is deferred
because filterOverflows and filterExpanded are bound in this view -
setting them inside the change-detection cycle would raise
ExpressionChangedAfterItHasBeenChecked.
Seeds HARDFORMULA/SOFTFORMULA columns on a newly-inserted row with
their computed formula string, the same way applyFormulaRules seeds
every row on initial load - a new row otherwise sits with a blank
formula cell until the next full reload. DC.ORIG_VALUE naturally
resolves to blank for this row (applyFormulaRules can't find a
dataSourceUnchanged match for a row that never existed before).
Written via hot.setDataAtRowProp (not a bare dataSource mutation, which
is what applyFormulaRules itself does) so HyperFormula's engine
actually learns about the new cell content - the same class of fix as
hot.alter() above, just for a single cell instead of a row structure
change.
Subscribes to the SAS VA data-driven content postMessage channel. VA pushes
a fresh message on every selection / parameter change. Registered early (in
ngOnInit) so the initial message isn't missed; messages are buffered on the
service and applied via a debounce so a burst of control clicks collapses
into a single reload on the latest message.
Sets or clears a single cell's "overwritten" comment, matching its
current value against dataSourceRaw (PK-matched, so this is safe to
call after a row insert/delete/sort has moved things around). No-op if
the row has no PK match in dataSourceRaw (a newly-inserted row) - see
findOverwrittenCells for why that's never revertable.
rowIndex is physical (matches dataSource's own order, same contract as
syncOverwrittenComments()'s own dataSource.forEach caller below) -
getDataAtRowProp and the comments plugin's own cell API all document a
VISUAL row instead, so it's translated via hot.toVisualRow() for every
Handsontable/plugin call, while dataSource itself stays indexed by the
physical rowIndex throughout.
Full resync of every revertable cell's "overwritten" comment against
the current data. Used both at initial load (nothing has a comment
yet, so this is purely additive) and after Cancel (an edit may have
just been discarded, so a previously-set comment can now be stale) -
Handsontable's comments plugin has no way to enumerate its own
comments, so the only way to find a stale one is to re-check every
candidate cell.
Toggles between live (auto-apply) and confirm filter modes. The choice is
stored on globals so it survives editor reloads. Switching to live with a
change still pending applies it immediately.
Recomputes this row's classification (M/A/D/U) and writes it into the
EDIT_STATUS cell - the same classification the row-header symbol shows,
but written into a real Handsontable cell so DC.ROW_STATUS-based
formulas can actually react to it. Written via hot.setDataAtRowProp
with a distinct source string (not a bare dataSource mutation) so
HyperFormula is notified and dependents recalculate - the source string
also lets the afterChange hook below recognise and ignore its own
writes, avoiding infinite recursion.
rowIndex is physical (matches dataSource's own order, same contract as
addRow()/insertRowAtPosition()'s own callers) - setDataAtRowProp
documents a VISUAL row instead, so it's translated via
hot.toVisualRow() right before that one call, while dataSource itself
stays indexed by the physical rowIndex.
Wraps hot.updateSettings() to work around a genuine Handsontable
multiColumnSorting + formulas plugin bug: calling updateSettings() -
with ANY settings, even a bare {} - while a sort is active corrupts
formula cell references, rendering '#REF!' instead of their computed
value (reproduced directly: a bare {} settings object alone is enough
to trigger it, independent of what it actually changes). Clearing the
sort first and restoring it immediately afterward avoids the
corruption entirely. Not needed for the very first updateSettings()
call at initial table load, since nothing can be sorted yet then.
editTable()/cancelEdit() deliberately do NOT call this wrapper - they
manage their own clear/restore inline, because their restore point is
later in the method (after other logic like syncOverwrittenComments()),
not immediately after their own updateSettings() call the way this
wrapper restores it. Don't "simplify" those two to use this instead
without preserving that later restore point.
The applied-filter panel is collapsed to a single line by default. The
chevron that expands it is only useful when the clause does not fit that
line, which depends on the rendered width - so it is measured from the DOM
rather than guessed from the length of the text.
Types of limitations ordered by priority of enforcement (Restrictions upper on the list cannot be un-restricted by lower types)
Restrict edit record feature with config in startupservice (comes from appService)
Restrict edit record feature and add row if demo and demo limits set as such - since demo is limited to less rows, and buttons which adds rows triggers error
Restrict add record feature and add row based on configuration of Column Level Security
Debounce window (ms) for VA messages. Sized so a burst of rapid control
clicks coalesces into a single apply (the last one) BEFORE any filter
reload. Note VA itself also throttles how often it posts, so this stacks
with that cadence.
VA filter mode. Live = auto-apply each VA change; confirm = stage the change
and wait for the user to click Apply. Backed by globals (toggled by the
Auto-apply checkbox) so it persists across the editor reloads a filter apply
triggers. Defaults to 'live'.
import {
AfterViewChecked,
AfterViewInit,
ChangeDetectorRef,
Component,
ElementRef,
OnDestroy,
OnInit,
QueryList,
ViewChild,
ViewChildren,
ViewEncapsulation
} from '@angular/core'
import { ActivatedRoute, Router } from '@angular/router'
import Handsontable, { CellRange } from 'handsontable'
import { Subject, Subscription } from 'rxjs'
import { sanitiseForSas } from '../shared/utils/sanitise'
import { SasStoreService } from '../services/sas-store.service'
type AOA = any[][]
import { HotTableComponent } from '@handsontable/angular-wrapper'
import { UploadFile } from '@sasjs/adapter'
import { isSpecialMissing } from '@sasjs/utils/input/validators'
import { CellValidationSource } from '../models/CellValidationSource'
import { FileUploader } from '../models/FileUploader.class'
import { FilterGroup, FilterQuery } from '../models/FilterQuery'
import { HotTableInterface } from '../models/HotTable.interface'
import { buildExportMenuItem } from '../shared/hot-export/hot-export.util'
import {
$DataFormats,
DSMeta,
EditorsGetDataServiceResponse,
Version
} from '../models/sas/editors-getdata.model'
import { DataFormat } from '../models/sas/common/DateFormat'
import { Approver, ExcelRule, QueryClause } from '../models/TableData'
import { QueryComponent } from '../query/query.component'
import { EventService } from '../services/event.service'
import { HelperService } from '../services/helper.service'
import { LoggerService } from '../services/logger.service'
import { SasService } from '../services/sas.service'
import { UserService } from '../shared/user.service'
import { applyFormulaRules } from '../shared/dc-validator/utils/applyFormulaRules'
import { expandCellRanges } from './utils/expandCellRanges'
import { preventMenuItemAutoClose } from './utils/preventMenuItemAutoClose'
import { findOverwrittenCells } from '../shared/dc-validator/utils/findOverwrittenCells'
import { getFormulaCellsToPreserveOnCancel } from '../shared/dc-validator/utils/getFormulaCellsToPreserveOnCancel'
import { getRevertableCols } from '../shared/dc-validator/utils/getRevertableCols'
import { resolveRevertedCellValue } from '../shared/dc-validator/utils/resolveRevertedCellValue'
import { getStableFormulaBaseCols } from '../shared/dc-validator/utils/getStableFormulaBaseCols'
import { syncOverwrittenCellComment } from '../shared/dc-validator/utils/syncOverwrittenCellComment'
import { parseFormulaRule } from '../shared/dc-validator/utils/parseFormulaRule'
import { substituteColumnReferences } from '../shared/dc-validator/utils/substituteColumnReferences'
import { DcValidator } from '../shared/dc-validator/dc-validator'
import { Col } from '../shared/dc-validator/models/col.model'
import { DcValidation } from '../shared/dc-validator/models/dc-validation.model'
import { DQRule } from '../shared/dc-validator/models/dq-rules.model'
import { getHotDataSchema } from '../shared/dc-validator/utils/getHotDataSchema'
import { excelRound } from '../shared/dc-validator/utils/excelRound'
import { isEmpty } from '../shared/dc-validator/utils/isEmpty'
import { hasFormulaRules } from '../shared/dc-validator/utils/hasFormulaRules'
import { getFormulaColumnNames } from '../shared/dc-validator/utils/getFormulaColumnNames'
import { isCharacterColumn } from '../shared/dc-validator/utils/isCharacterColumn'
import { escapeCharacterColumnValue } from '../shared/dc-validator/utils/escapeCharacterColumnValue'
import { unescapeFormula } from '../shared/dc-validator/utils/unescapeFormula'
import {
AutoEscapedCellMap,
getRowKey,
markAutoEscaped,
isAutoEscaped,
clearAutoEscaped
} from '../shared/dc-validator/utils/autoEscapedCellTracker'
import { HyperFormula } from 'hyperformula'
import { parseLabelsParam } from '../shared/utils/parse-labels-param'
import { getDisplayColHeaders } from '../shared/utils/display-col-headers'
import { buildColInfoHtml } from '../shared/utils/col-info-html'
import { globals } from '../_globals'
import { UploadStaterComponent } from './components/upload-stater/upload-stater.component'
import {
DynamicCellValidation,
DynamicExtendedCellValidation
} from './models/dynamicExtendedCellValidation'
import { EditRecordInputFocusedEvent } from './models/edit-record/edit-record-events'
import { EditorRestrictions } from './models/editor-restrictions.model'
import { parseTableColumns } from './utils/grid.utils'
import { classifyRow } from './utils/classifyRow'
import { withoutEditStatus } from './utils/withoutEditStatus'
import { getEditStatusSymbol } from './utils/getEditStatusSymbol'
import { normalizeSortConfig } from './utils/normalizeSortConfig'
import { EDIT_STATUS_COLUMN_NAME } from '../shared/dc-validator/utils/editStatusColumnRule'
import { DELETE_RECORD_COLUMN_NAME } from '../shared/dc-validator/utils/deleteRecordColumnRule'
import {
prepareDeleteRecordColumn,
seedDeleteRecordColumn
} from '../shared/dc-validator/utils/deleteRecordColumn'
import {
errorRenderer,
noSpinnerRenderer,
spinnerRenderer
} from './utils/renderers.utils'
import { LicenceService } from '../services/licence.service'
import { FileUploadEncoding } from '../models/FileUploadEncoding'
import { SpreadsheetService } from '../services/spreadsheet.service'
import { VaMessagingService, VaMessage } from '../services/va-messaging.service'
import { VaFilterService } from '../services/va-filter.service'
import { UploadFileResponse } from '../models/UploadFile'
import { RequestWrapperResponse } from '../models/request-wrapper/RequestWrapperResponse'
import { ParseResult } from '../models/ParseResult.interface'
@Component({
selector: 'app-editor',
templateUrl: './editor.component.html',
styleUrls: ['./editor.component.scss'],
host: {
class: 'content-container'
},
encapsulation: ViewEncapsulation.None,
standalone: false
})
export class EditorComponent
implements OnInit, AfterViewInit, AfterViewChecked, OnDestroy
{
@ViewChildren('uploadStater')
uploadStaterCompList: QueryList<UploadStaterComponent> = new QueryList()
@ViewChildren('queryFilter')
queryFilterCompList: QueryList<QueryComponent> = new QueryList()
@ViewChild(HotTableComponent, { static: false })
hotTableComponent!: HotTableComponent
@ViewChildren('fileUploadInput')
fileUploadInputCompList: QueryList<ElementRef> = new QueryList()
public static cnt = 0
public static nonPkCnt = 0
public static lastCell = 0
private _tableSub: Subscription | undefined
public message = ''
public $dataFormats: $DataFormats | null = null
public submit: boolean | undefined
public cols: Col[] = []
@ViewChild('ht', { static: true }) ht!: ElementRef
/** Feature restrictions
*
* What can be restricted
* - Add Row button
* - Insert rows above/below
* - Add record button
* - Edit record button
*
* Types of limitations ordered by priority of enforcement (Restrictions upper on the list cannot be un-restricted by lower types)
* - Restrict edit record feature with config in startupservice (comes from appService)
* - Restrict `edit record feature` and `add row` if demo and demo limits set as such - since demo is limited to less rows, and buttons which adds rows triggers error
* - Restrict `add record feature` and `add row` based on configuration of `Column Level Security`
*/
restrictions: EditorRestrictions = {}
datasetInfo = false
dsmeta: DSMeta[] = []
versions: Version[] = []
dsNote = ''
/** Set when getdata served a truncated selection - shown as a warning
* against the table name, with the reason in a tooltip. */
truncationNote = ''
viewboxes = false
Infinity = Infinity
public hotInstance!: Handsontable
public dcValidator: DcValidator | undefined
public hotTableSettings: Handsontable.GridSettings = {}
private updateHotTableSettings(): void {
this.hotTableSettings = {
colHeaders: this.hotTable.colHeaders,
columns: this.hotTable.columns,
height: this.hotTable.height,
licenseKey: this.hotTable.licenseKey,
readOnly: this.hotTable.readOnly,
copyPaste: this.hotTable.copyPaste,
// Must carry the current formulas config through - this rebuilds
// hotTableSettings (and re-applies it via the [settings] binding)
// whenever hot_license_key emits, which can happen after a table has
// already loaded and turned formulas on; omitting the key here would
// let that rebuild silently drop it again.
formulas: this.hotTable.formulas,
contextMenu: true,
className: 'htDark',
theme: 'ht-theme-classic'
}
}
public hotTable: HotTableInterface = {
data: [],
colHeaders: [],
hidden: true,
columns: [],
height: 'calc(100vh - 160px)',
licenseKey: undefined,
readOnly: true,
// Every table gets the formulas engine, regardless of whether it
// declares any HARDFORMULA/SOFTFORMULA rules - set here (not only
// inside initSetup) so it's already present in hotTableSettings for
// the very first [settings] binding, before any table has loaded.
formulas: { engine: HyperFormula, licenseKey: 'gpl-v3' },
copyPaste: {
copyColumnHeaders: true,
copyColumnHeadersOnly: true
},
settings: {
contextMenu: {
items: {
edit_row: {
name: 'Edit row',
// HOT 18's MenuItemConfig types `hidden` as a plain `() => boolean` with
// no `this` type, so an object-literal method here has `this` inferred
// as the surrounding item config unless declared explicitly.
hidden(this: Handsontable.Core) {
const hot = this
// Hide editing actions in read-only (view) mode.
if (hot.getSettings().readOnly) return true
const fullCellRange: CellRange[] | undefined =
hot.getSelectedRange()
if (!fullCellRange) return false
const cellRange = fullCellRange[0]
return cellRange.from.row !== cellRange.to.row
},
callback: (
key: string,
selection: any[],
clickEvent: MouseEvent
) => {
const firstSelection = selection[0]
if (firstSelection.start.row === firstSelection.end.row) {
this.editRecord(null, firstSelection.start.row)
}
}
},
// Only ever shown when the selection contains at least one cell
// markOverwrittenCells (or the live afterChange sync) attached a
// comment to - i.e. a cell whose current value differs from what
// SAS actually sent for it, formula-caused or a direct edit. The
// comment's presence is a sufficient and exact signal, no need to
// separately recompute "is this overwritten" here too. The
// selection can be a single cell, a rectangular multi-cell range,
// a whole row/column (header click), or several disjoint ranges
// (ctrl-click) - expandCellRanges normalizes all of those into a
// flat list of individual cells.
revert_cells: {
name: 'Revert',
hidden(this: Handsontable.Core) {
if (this.getSettings().readOnly) return true
const ranges: CellRange[] | undefined = this.getSelectedRange()
if (!ranges || ranges.length === 0) return true
const commentsPlugin: any = this.getPlugin('comments')
const cells = expandCellRanges(
ranges,
this.countRows(),
this.countCols()
)
return !cells.some(({ row, col }) =>
commentsPlugin.getCommentAtCell(row, col)
)
},
callback: (key: string, selection: any[]) => {
const hot = this.hotInstance
const commentsPlugin: any = hot.getPlugin('comments')
const cells = expandCellRanges(
selection.map((sel) => ({ from: sel.start, to: sel.end })),
hot.countRows(),
hot.countCols()
)
for (const { row, col } of cells) {
const comment: string | undefined =
commentsPlugin.getCommentAtCell(row, col)
if (!comment) continue
const prop = hot.colToProp(col) as string
const rawValueText = comment.slice(
EditorComponent.ORIGINAL_VALUE_COMMENT_PREFIX.length
)
const isNumericCol =
this.dcValidator?.getRule(prop)?.type === 'numeric'
hot.setDataAtRowProp(
row,
prop,
resolveRevertedCellValue(rawValueText, isNumericCol),
'revert'
)
commentsPlugin.removeCommentAtCell(row, col)
}
}
},
// Only ever shown when the selection contains at least one cell
// this app itself auto-escaped (see character-column-formula-
// plan.md) - a genuinely literal leading `'` the backend sent, or
// one the user typed themselves, is never in autoEscapedCells and
// so never makes this item appear or gets touched by it.
apply_as_formula: {
name: 'Apply as formula',
hidden: (): boolean => {
const hot = this.hotInstance
if (hot.getSettings().readOnly) return true
const ranges: CellRange[] | undefined = hot.getSelectedRange()
if (!ranges || ranges.length === 0) return true
const cells = expandCellRanges(
ranges,
hot.countRows(),
hot.countCols()
)
return !cells.some(({ row, col }) => {
const dataRow = this.dataSource[hot.toPhysicalRow(row)]
if (!dataRow) return false
const colName = hot.colToProp(col) as string
return isAutoEscaped(
this.autoEscapedCells,
getRowKey(dataRow, this.headerPks),
colName
)
})
},
callback: (key: string, selection: any[]) => {
const hot = this.hotInstance
const cells = expandCellRanges(
selection.map((sel) => ({ from: sel.start, to: sel.end })),
hot.countRows(),
hot.countCols()
)
for (const { row, col } of cells) {
const dataRow = this.dataSource[hot.toPhysicalRow(row)]
if (!dataRow) continue
const colName = hot.colToProp(col) as string
const rowKey = getRowKey(dataRow, this.headerPks)
if (!isAutoEscaped(this.autoEscapedCells, rowKey, colName))
continue
hot.setDataAtRowProp(
row,
colName,
unescapeFormula(hot.getDataAtRowProp(row, colName))
)
clearAutoEscaped(this.autoEscapedCells, rowKey, colName)
}
}
},
row_above: {
name: 'Insert Row above',
hidden: () => this.hotTable.readOnly === true,
callback: (
key: string,
selection: any[],
clickEvent: MouseEvent
) => {
const firstSelection = selection[0]
const targetRow = firstSelection.start.row
this.insertRowAtPosition(targetRow)
}
},
row_below: {
name: 'Insert Row below',
hidden: () => this.hotTable.readOnly === true,
callback: (
key: string,
selection: any[],
clickEvent: MouseEvent
) => {
const firstSelection = selection[0]
const targetRow = firstSelection.start.row + 1
this.insertRowAtPosition(targetRow)
}
},
remove_row: {
name: 'Ignore row',
hidden: () => this.hotTable.readOnly === true
},
copy: {
name: 'Copy without headers'
},
copy_with_column_headers: {
name: 'Copy with headers'
},
copy_column_headers_only: {
name: 'Copy headers only'
},
// Client-side export of the current grid view (or selection).
// skipLeadingCols=1 drops the `Delete?` housekeeping column (col 0).
export_file: buildExportMenuItem(() => this.libds || 'export', 1),
sp1: {
name: '---------',
hidden: () => this.hotTable.readOnly === true
},
undo: {
name: 'Undo',
hidden: () => this.hotTable.readOnly === true
},
redo: {
name: 'Redo',
hidden: () => this.hotTable.readOnly === true
},
// Navigates rather than just flipping `useLabels` locally, so the
// URL stays the single source of truth (shareable/refreshable
// state) — the queryParams subscription in ngOnInit picks up the
// change and re-renders colHeaders.
toggle_labels: {
name: () => (this.useLabels ? 'Show names' : 'Show labels'),
callback: () => {
this.router.navigate([], {
relativeTo: this.route,
queryParams: { labels: this.useLabels ? null : true },
queryParamsHandling: 'merge'
})
}
}
}
}
}
}
public hotCellsPropRow: number | null = null
public filter = false
public submitLoading = false
public uploadLoading = false
public rowsChanged: any = {
rowsUpdated: 0,
rowsDeleted: 0,
rowsAdded: 0
}
public modifedRowsIndexes: number[] = []
public queryErr = false
public queryErrMessage: string | undefined
public successEnable = false
public libTab: string | undefined
public queryFilter: any
public _query: Subscription | undefined
private _queryParams: Subscription | undefined
// `?labels=true` toggle (see parseLabelsParam) — URL is the source of
// truth, kept in sync via the queryParams subscription in ngOnInit.
public useLabels: boolean = false
public whereString: string | undefined
public clauses: any
public nullVariables = false
public tableId: string | undefined
public pkFields: any = []
public libds: string | undefined
public filter_pk: string | undefined
public table: any
public filename = ''
public selectedColumn: any
public hotSelection: Array<number> | null | undefined
public submitLimitNotice = false
public badEdit = false
public badEditCause: string | undefined
public badEditTitle: string | undefined
get embed() {
return globals.embed
}
/** True when running as a SAS Visual Analytics data-driven content object. */
get isVaEmbed() {
return globals.embed === 'va'
}
/**
* VA filter mode. Live = auto-apply each VA change; confirm = stage the change
* and wait for the user to click Apply. Backed by globals (toggled by the
* Auto-apply checkbox) so it persists across the editor reloads a filter apply
* triggers. Defaults to 'live'.
*/
get vaAutoApply(): boolean {
return globals.vaApplyMode !== 'confirm'
}
/**
* VA filter UI status (shown in both modes):
* - 'pending' : a VA change was received and not yet fetched (debouncing in
* live mode, or awaiting Apply in confirm mode);
* - 'loading' : the filter is being fetched (saveQuery), just before DC's
* native "Loading Table" takes over for the data;
* - 'idle' : nothing pending.
*/
public vaFilterStatus: 'idle' | 'pending' | 'loading' = 'idle'
/** The staged clauses/signature for the pending VA filter. */
private vaPendingClauses: QueryClause[] | null = null
private vaPendingSignature = ''
/** Removes the VA postMessage listener; set while subscribed. */
private vaUnsubscribe?: () => void
/** Guards against overlapping VA filter reloads. */
private vaApplyingFilter = false
/** Debounce timer coalescing a burst of VA messages into one apply. */
private vaDebounceTimer?: ReturnType<typeof setTimeout>
/**
* Debounce window (ms) for VA messages. Sized so a burst of rapid control
* clicks coalesces into a single apply (the last one) BEFORE any filter
* reload. Note VA itself also throttles how often it posts, so this stacks
* with that cadence.
*/
private static readonly VA_DEBOUNCE_MS = 800
// Shared between markOverwrittenCells/syncOverwrittenCommentForCell
// (writes it) and the revert_cells context menu item (parses it back
// out) - see findOverwrittenCells.
private static readonly ORIGINAL_VALUE_COMMENT_PREFIX = 'Original value: '
public tableTrue: boolean | undefined
public saveLoading = false
public approvers: string[] = []
public approver: any
public readOnlyFields!: number
public errValidation = false
public dataObj: any
public disableSubmit: boolean | undefined
public pkNull = false
public noPkNull = false
public tableData: Array<any> = []
public queryText = ''
public queryTextSaved = ''
/**
* The applied-filter panel is collapsed to a single line by default. The
* chevron that expands it is only useful when the clause does not fit that
* line, which depends on the rendered width - so it is measured from the DOM
* rather than guessed from the length of the text.
*/
public filterExpanded = false
public filterOverflows = false
@ViewChild('infoBar') infoBar?: ElementRef<HTMLElement>
private filterMeasuredKey = ''
public showApprovers = false
public pkDups = false
public validationDone = 0
public duplicatePkIndexes: any = []
public columnHeader: string[] = []
public specInfo: { col: string; len: number; type: number }[] = []
public tooLong = false
public exceedCells: {
col: string
len: number
val: string
}[] = []
public uploader: FileUploader = new FileUploader()
public uploadUrl = ''
public excelFileReady = false
public uploadPreview = false
public excelFileParsing = false
public excelUploadState: string | null = null
public data: AOA = []
public headerArray: string[] = []
public hotDataSchema: any = {}
public headerShow: string[] = []
public headerVisible = false
public hasBaseDropZoneOver = false
public hasAnotherDropZoneOver = false
public headerPks: string[] = []
/** Derived (computed) columns from MPE_DERIVED - read-only, and marked as such */
public derivedVars: string[] = []
public columnLevelSecurityFlag = false
public dateTimeHeaders: string[] = []
public timeHeaders: string[] = []
public dateHeaders: string[] = []
public xlRules: ExcelRule[] = []
public encoding: FileUploadEncoding = 'UTF-8'
// header column names
headerColumns: Array<any> = []
cellValidation: DcValidation[] = []
// hot table data source
dataSource!: any[]
prevDataSource!: any[]
dataSourceUnchanged!: any[]
// Raw, as-received-from-SAS snapshot, captured before applyFormulaRules
// overwrites HARDFORMULA/SOFTFORMULA columns - see findOverwrittenCells
// and getRevertableColumnNames. Distinct from dataSourceUnchanged, which is a
// per-editing-session baseline that gets reset on every editTable() call;
// this stays fixed for the table's whole lifetime, since "what did the
// real dataset actually have before any formula got involved" doesn't
// change across edit sessions.
dataSourceRaw!: any[]
dataSourceBeforeSubmit!: any[]
dataModified!: any[]
// Cells the app itself auto-escaped (see escapeCharacterColumnValue) -
// PK-keyed, not a property on the row objects, so it can never leak into
// the submit payload and survives insert/delete/sort. Reset on every
// fresh load, same lifecycle as dataSourceRaw/dataSourceUnchanged. See
// character-column-formula-plan.md.
private autoEscapedCells: AutoEscapedCellMap = new Map()
public filePasswordSubject: Subject<string | undefined> = new Subject()
public fileUnlockError = false
public filePasswordModal = false
public showUploadModal = false
public discardSourceFile = false
public manualFileEditModal = false
public recordAction: string | null = null
public currentEditRecord: any
public currentEditRecordValidator: DcValidator | undefined
public currentEditRecordLoadings: number[] = []
public currentEditRecordErrors: number[] = []
public currentEditRecordIndex = -1
public generateEditRecordUrlLoading = false
public generatedRecordUrl: string | null = null
public addRecordUrl: string | null = null
public recordNewOrPkModified = false
public addRecordLoading = false
public singleRowSelected = false
public addingNewRow = false
public getdataError = false
public zeroFilterRows = false
public tableFileDragOver = false
/**
* Hash/values table used for dynamic cell validation
*/
public cellValidationSource: CellValidationSource[] = []
public validationTableLimit = 100
// Incremented on cancel/edit-exit so in-flight dynamic-validation
// responses can detect they should drop their post-response work.
private validationEpoch = 0
// Cells currently showing the loading spinner renderer (keyed `r,c`),
// so cancelBulkValidation can reset them.
private pendingSpinnerCells = new Set<string>()
// State for the bulk-validation progress banner (paste / autofill).
public bulkValidation: {
active: boolean
done: number
total: number
} = { active: false, done: 0, total: 0 }
// Confirm-modal state used to gate large paste validations.
public confirmModal: {
open: boolean
title: string
message: string
} = { open: false, title: '', message: '' }
private confirmModalResolver: ((v: boolean) => void) | null = null
public disabledBasicDynamicCellValidationMap: {
row: number
col: number
active: boolean
}[] = []
public licenceState = this.licenceService.licenceState
private ariaObserver: MutationObserver | undefined
private ariaCheckInterval: any | undefined
private gridResizeObserver: ResizeObserver | undefined
private pasteListener: ((event: ClipboardEvent) => void) | undefined
constructor(
private licenceService: LicenceService,
private eventService: EventService,
private loggerService: LoggerService,
private sasStoreService: SasStoreService,
private helperService: HelperService,
private router: Router,
private route: ActivatedRoute,
private sasService: SasService,
private cdf: ChangeDetectorRef,
private spreadsheetService: SpreadsheetService,
private vaMessaging: VaMessagingService,
private vaFilter: VaFilterService,
private userService: UserService
) {
this.parseRestrictions()
this.setRestrictions()
}
/**
* Prepare feature restrictions based on licence key
*/
private parseRestrictions() {
this.restrictions.restrictAddRecord =
this.licenceState.value.addRecord === false
this.restrictions.restrictEditRecord =
this.licenceState.value.editRecord === false
this.restrictions.restrictFileUpload =
this.licenceState.value.fileUpload === false
}
/**
* Applying prepared restrictions
* @param overrideRestrictions can be used to apply and override specific restrictions
*/
private setRestrictions(overrideRestrictions?: EditorRestrictions) {
if (overrideRestrictions) {
this.restrictions = {
...this.restrictions,
...overrideRestrictions
}
}
if (this.restrictions.removeEditRecordButton) {
delete (this.hotTable?.settings?.contextMenu as any).items.edit_row
}
if (this.restrictions.restrictAddRow) {
delete (this.hotTable?.settings?.contextMenu as any).items.row_above
delete (this.hotTable?.settings?.contextMenu as any).items.row_below
delete (this.hotTable?.settings?.contextMenu as any).items.remove_row
}
}
/**
* Disabling add row button based on wether rows limit is present
*/
private checkRowLimit() {
if (this.columnLevelSecurityFlag) return
if (this.licenceState.value.editor_rows_allowed !== Infinity) {
if (
this.dataSource?.length >= this.licenceState.value.editor_rows_allowed
) {
this.restrictions.restrictAddRow = true
} else {
this.restrictions.restrictAddRow = false
}
}
}
/**
* Resetting filter variables
*/
public resetFilter() {
if (this.queryFilterCompList.first) {
this.queryFilterCompList.first.resetFilter()
}
}
/**
* Openning file upload modal
* If feature is locked, `feature locked` modal will be shown
*/
public onShowUploadModal() {
if (this.restrictions.restrictFileUpload) {
this.eventService.showDemoLimitModal('File Upload')
return
}
if (this.columnLevelSecurityFlag) {
this.eventService.showInfoModal(
'Information',
'Upload feature is disabled while Column Level Security rules are active'
)
return
}
if (!this.uploadPreview) this.showUploadModal = true
}
/**
* Called by FileDropDirective
* @param e true if file is dragged over the drop zone
*/
public fileOverBase(e: boolean): void {
this.hasBaseDropZoneOver = e
}
public attachFile(event: any, dropped = false) {
const file: File = dropped ? event[0] : event.target.files[0]
this.excelUploadState = 'Loading'
this.excelFileParsing = true
this.excelFileReady = false
this.filename = file.name
this.spreadsheetService
.parseExcelFile(
{
file: file,
uploader: this.uploader,
dcValidator: this.dcValidator!,
headerPks: this.headerPks,
headerArray: this.headerArray,
headerShow: this.headerShow,
timeHeaders: this.timeHeaders,
dateHeaders: this.dateHeaders,
dateTimeHeaders: this.dateTimeHeaders,
xlRules: this.xlRules,
encoding: this.encoding
},
(uploadState: string) => {
this.appendUploadState(uploadState)
},
(tableFoundInfo: string) => {
this.eventService.showInfoModal('Table Found', tableFoundInfo)
}
)
.then(async (parseResult: ParseResult | undefined) => {
if (parseResult) {
this.excelFileReady = true
this.uploader = parseResult.uploader
if (parseResult.data && parseResult.headerShow) {
// If data is returned it means we parsed excel file
this.data = parseResult.data
this.headerShow = parseResult.headerShow
this.getPendingExcelPreview()
} else {
// otherwise it's csv file, and we send them directly
await this.uploadParsedFiles()
}
}
})
.catch((error: string) => {
this.eventService.showAbortModal(null, error, null)
this.showUploadModal = false
this.uploadPreview = false
setTimeout(() => {
this.filename = ''
})
})
.finally(() => {
this.excelFileParsing = false
})
}
/**
* Submits attached excel file that is in preview mode
*/
public submitExcel() {
if (this.licenceState.value.submit_rows_limit !== Infinity) {
this.submitLimitNotice = true
return
}
this.uploadParsedFiles()
}
/**
* This method will run validations and upload all of the pending files
* that are in the uploader queue.
*/
public async uploadParsedFiles() {
if (this.checkInvalid()) {
this.eventService.showAbortModal(null, 'Invalid values are present.')
return
}
this.validatePrimaryKeys()
if (this.duplicatePkIndexes.length !== 0) {
this.pkDups = true
this.submit = false
return
} else {
this.pkDups = false
}
this.uploadLoading = true
const filesToUpload: UploadFile[] = []
for (const file of this.uploader.queue) {
filesToUpload.push({
file: file,
fileName: file.name
})
}
await this.sasService
.uploadFile(this.uploadUrl, filesToUpload, { table: this.libds })
.then(
(res: UploadFileResponse) => {
if (typeof res.adapterResponse.sasjsAbort === 'undefined') {
if (typeof res.adapterResponse.sasparams === 'undefined') {
return
} else {
this.uploadLoading = false
const params = res.adapterResponse.sasparams[0]
this.successEnable = true
this.tableId = params.DSID
this.router.navigateByUrl('/stage/' + this.tableId)
}
} else {
// handle succesfull response
const abortRes = res.adapterResponse
const abortMsg = abortRes.sasjsAbort[0].MSG
const macMsg = abortRes.sasjsAbort[0].MAC
this.uploadLoading = false
this.filename = ''
if (this.fileUploadInputCompList.first) {
//clear the attached file to input
this.fileUploadInputCompList.first.nativeElement.value = ''
}
this.uploader.queue = []
this.eventService.showAbortModal('', abortMsg, {
SYSWARNINGTEXT: abortRes.SYSWARNINGTEXT,
SYSERRORTEXT: abortRes.SYSERRORTEXT,
MAC: macMsg,
_PROGRAM: abortRes._PROGRAM
})
}
},
(err: any) => {
this.uploadLoading = false
if (this.fileUploadInputCompList.first) {
//clear the attached file to input
this.fileUploadInputCompList.first.nativeElement.value = ''
}
this.uploader.queue = []
this.eventService.catchResponseError(
'file upload',
err.adapterResponse
)
}
)
}
/**
* After excel file is attached and parsed, this function will display it's content in the HOT table in read only mode
*/
public getPendingExcelPreview() {
this.queryTextSaved = this.queryText
this.queryText = ''
this.excelUploadState = 'Parsing'
this.toggleHotPlugin('contextMenu', false)
const previewDatasource: any[] = []
this.data.map((item) => {
const itemObject: any = {}
this.headerShow.map((header: any, index: number) => {
itemObject[header] = item[index]
})
// If Delete? column is not set in the file, we set it to NO
if (!itemObject['_____DELETE__THIS__RECORD_____'])
itemObject['_____DELETE__THIS__RECORD_____'] = 'No'
// EDIT_STATUS is never part of the uploaded file (client-only, see
// editStatusColumnRule.ts) - default it the same as a fresh load.
itemObject[EDIT_STATUS_COLUMN_NAME] = 'U'
previewDatasource.push(itemObject)
})
// An uploaded file is bulk, external data - not something the user
// typed/pasted cell-by-cell - so a `=`-led value in a character column
// must be escaped to inert text, the same way a fresh backend load is
// (see initSetup's own escape loop). Formulas are enabled for every
// table, so without this a spreadsheet cell that merely looks like a
// formula (an inventory code, a serial number) would silently start
// evaluating as a live one.
const formulaBaseCols = getFormulaColumnNames(
this.dcValidator?.getDqDetails() ?? []
)
this.autoEscapedCells = new Map()
for (const row of previewDatasource) {
const rowKey = getRowKey(row, this.headerPks)
for (const colName of this.headerColumns) {
if (formulaBaseCols.includes(colName)) continue
if (!isCharacterColumn(this.cols, colName)) continue
const { value, wasEscaped } = escapeCharacterColumnValue(row[colName])
// A row whose backend source never had this column at all (a
// sparse row) must stay that way - writing back unconditionally
// would materialize a new own property set to undefined, which
// the SASjs adapter rejects at submit time even though nothing
// about this cell ever changed.
if (!wasEscaped) continue
row[colName] = value
markAutoEscaped(this.autoEscapedCells, rowKey, colName)
}
}
this.dataSourceUnchanged = this.helperService.deepClone(this.dataSource)
this.dataSource = previewDatasource
this.hotTable.data = previewDatasource
const hot = this.hotInstance
this.excelUploadState = 'Validating-HOT'
this.updateSettingsSortSafe(
{
data: this.dataSource,
maxRows: Infinity
},
false
)
hot.render()
this.appendUploadState(`Validating rows`)
hot.validateCells(() => {
this.showUploadModal = false
this.uploadPreview = true
this.excelFileParsing = false
this.excelUploadState = null
})
}
/**
* Drops the attached excel file
* @param discardData wheter to discard data parsed from the file or to keep it in the table after dropping a attached excel file
*/
public discardPendingExcel(discardData?: boolean) {
this.updateSettingsSortSafe({
maxRows: this.licenceState.value.editor_rows_allowed
})
if (discardData) this.cancelEdit()
if (this.fileUploadInputCompList.first) {
this.fileUploadInputCompList.first.nativeElement.value = ''
}
this.uploadPreview = false
this.excelFileReady = false
this.uploader.queue = []
if (!isNaN(parseInt(this.router.url.split('/').pop() || ''))) {
if (this.queryTextSaved.length > 0) {
this.queryText = this.queryTextSaved
this.queryTextSaved = ''
}
}
}
/**
* Drops attached excel file, keeps it's data in the DC table
* User can now edit the table and submit. Witout the file present.
*/
public previewTableEditConfirm() {
this.discardPendingExcel()
this.convertToCorrectTypes(this.dataSource)
this.editTable(true)
}
private appendUploadState(state: string, replaceLast = false) {
this.cdf.detectChanges()
if (this.uploadStaterCompList.first) {
if (replaceLast) {
this.uploadStaterCompList.first.replaceLastState(state)
} else {
this.uploadStaterCompList.first.appendState(state)
}
}
}
isColPk(col: string) {
return this.headerPks.indexOf(col) > -1
}
isReadonlyCol(col: string | number) {
const colRules = this.dcValidator?.getRule(col)
return colRules?.readOnly
}
/**
* A derived (computed) column - configured in MPE_DERIVED, created at read
* time. Read-only, and styled apart from an ordinary read-only column so the
* user can see that there is nothing to edit rather than that they may not.
*/
isDerivedCol(col: string | number) {
const name =
typeof col === 'number'
? (this.hotInstance?.colToProp(col) as string) || this.columnHeader[col]
: col
return !!name && this.derivedVars.includes(name.toUpperCase())
}
isColHeader(col: string) {
return this.headerArray.indexOf(col.toUpperCase()) > -1
}
removeQuery() {
this.sasStoreService.removeClause()
}
async sendClause() {
this.submitLoading = true
let nullVariableArr = []
const emptyVariablesArr = []
// to check number of empty clauses
if (typeof this.clauses === 'undefined') {
this.nullVariables = true
this.submitLoading = false
return
} else {
const query = this.clauses.queryObj
if (query[0].elements.length < 1) {
// Clear cached filtering data
if (globals.rootParam === 'home' || globals.rootParam === 'editor') {
globals.editor.filter.clauses = []
globals.editor.filter.query = []
globals.editor.filter.groupLogic = ''
}
// Reset filtering
this.router.navigate(['/editor/' + this.libds], {
queryParamsHandling: 'preserve'
})
return
}
for (let index = 0; index < query.length; index++) {
const el = query[index].elements
nullVariableArr = el.filter(function (item: any) {
return item.variable === null
})
if (nullVariableArr.length) {
emptyVariablesArr.push(el)
}
}
}
if (emptyVariablesArr.length) {
this.nullVariables = true
this.submitLoading = false
return
} else {
try {
if (this.clauses !== undefined && this.libds) {
const filterQuery: FilterQuery = {
groupLogic: this.clauses.groupLogic,
filterGroups: []
}
this.clauses.queryObj.forEach((group: any) => {
const filterGroup: FilterGroup = {
filterClauses: []
}
group.elements.forEach((clause: any) => {
filterGroup.filterClauses.push(
this.helperService.deepClone(clause)
)
})
filterGroup.clauseLogic = group.clauseLogic
filterQuery.filterGroups.push(
this.helperService.deepClone(filterGroup)
)
})
const filterQueryClauseTable =
this.sasStoreService.createFilterQueryTable(filterQuery)
await this.sasStoreService
.saveQuery(this.libds, filterQueryClauseTable)
.then((res: any) => {
const id = res.result[0].FILTER_RK
const table = res.result[0].FILTER_TABLE
this.queryFilter = { id: id, table: table }
this.router
.navigate(['/'], {
skipLocationChange: true,
queryParamsHandling: 'preserve'
})
.then(() =>
this.router.navigate(
[
'/editor/' +
this.queryFilter.table +
'/' +
this.queryFilter.id
],
{
queryParamsHandling: 'preserve'
}
)
)
this.filter = false
})
.catch((err: any) => {
this.submitLoading = false
})
}
} catch (error: any) {
this.queryErr = true
this.submitLoading = false
this.queryErrMessage = error
}
}
}
public openQb() {
if (this.libds) {
// this.libTab = this.libds;
this.filter = true
this.cdf.detectChanges()
this.submitLoading = false
// The columns that came with the table are the cheapest source for the
// picker. When the table was not returned - the row/cell limits abort the
// request before the data arrives - that list is empty, so the store
// falls back to public/getcols rather than leaving the picker with
// nothing to pick from.
this.sasStoreService
.setQueryVariablesFromTable(this.libds, this.cols)
.catch((err: any) =>
this.eventService.catchResponseError('public/getcols', err)
)
}
}
/**
* Reads the current multi-column sort config defensively. The sorting plugin
* may not be initialised yet (e.g. editTable() invoked right after load in VA
* embed mode, before the grid finished setup), in which case getSortConfig()
* throws on null internal state. There's no sort to preserve then, so [].
*/
private getCurrentSortConfigs(): any[] {
const hot = this.hotInstance
if (!hot) return []
try {
const plugin = hot.getPlugin('multiColumnSorting')
return normalizeSortConfig(plugin?.getSortConfig())
} catch {
return []
}
}
/**
* Wraps hot.updateSettings() to work around a genuine Handsontable
* multiColumnSorting + formulas plugin bug: calling updateSettings() -
* with ANY settings, even a bare {} - while a sort is active corrupts
* formula cell references, rendering '#REF!' instead of their computed
* value (reproduced directly: a bare {} settings object alone is enough
* to trigger it, independent of what it actually changes). Clearing the
* sort first and restoring it immediately afterward avoids the
* corruption entirely. Not needed for the very first updateSettings()
* call at initial table load, since nothing can be sorted yet then.
*
* editTable()/cancelEdit() deliberately do NOT call this wrapper - they
* manage their own clear/restore inline, because their restore point is
* later in the method (after other logic like syncOverwrittenComments()),
* not immediately after their own updateSettings() call the way this
* wrapper restores it. Don't "simplify" those two to use this instead
* without preserving that later restore point.
*/
private updateSettingsSortSafe(
settings: Handsontable.GridSettings,
render?: boolean
): void {
const hot = this.hotInstance
const columnSorting = hot.getPlugin('multiColumnSorting')
const sortConfigs = this.getCurrentSortConfigs()
if (sortConfigs.length > 0) columnSorting.clearSort()
hot.updateSettings(settings, render)
// sort() replaces the entire sort state on every call rather than
// accumulating (see its own JSDoc) - restoring a multi-column sort
// needs one call with the whole array, not a loop of single-column
// calls, or every column but the last silently loses its sort.
if (sortConfigs.length > 0) columnSorting.sort(sortConfigs)
}
editTable(previewEdit?: boolean, newRow?: boolean) {
this.toggleHotPlugin('contextMenu', true)
const hot = this.hotInstance
if (!hot) return
// Entering edit mode: cancel any scheduled VA filter apply so it can't fire
// mid-edit and wipe the edits. A pending filter is re-applied on the return
// to read-only (cancelEdit).
if (this.vaDebounceTimer) {
clearTimeout(this.vaDebounceTimer)
this.vaDebounceTimer = undefined
}
const columnSorting = hot.getPlugin('multiColumnSorting')
const sortConfigs = this.getCurrentSortConfigs()
setTimeout(() => {
if (!previewEdit) {
this.dataSourceUnchanged = this.helperService.deepClone(this.dataSource)
if (newRow) {
this.dataSourceUnchanged.pop()
}
this.overlayFormulaRawValuesOnUnchanged(this.dataSourceUnchanged)
}
this.hotTable.readOnly = false
this.hotTable.data = this.dataSource
// Handsontable's multiColumnSorting + formulas integration corrupts
// formula cell references (renders '#REF!') if updateSettings() runs
// while a sort is still active, even for an unrelated setting -
// clear it first and restore the captured sortConfigs afterward
// (below), rather than leaving the sort applied across the call.
if (sortConfigs.length > 0) columnSorting.clearSort()
hot.updateSettings(
{
readOnly: this.hotTable.readOnly
},
false
)
hot.render()
// Resync every revertable cell's "overwritten" comment against
// dataSourceRaw on every entry into edit mode. Needed for the Excel
// upload path in particular (previewTableEditConfirm -> editTable(true))
// - the preview's bulk hot.updateSettings({data: ...}) fires afterChange
// with source 'loadData', which the live sync hook deliberately ignores
// (see its own comment), so an uploaded value that differs from the
// original SAS data would otherwise never get flagged/commented. A
// plain Edit-button click re-syncs the same (already-correct) state,
// same as cancelEdit() does for the read-only return path.
this.syncOverwrittenComments()
// sort() replaces the entire sort state on every call rather than
// accumulating (see its own JSDoc) - a multi-column sort needs one
// call with the whole array, not a loop of single-column calls, or
// every column but the last silently loses its sort.
if (sortConfigs.length > 0) columnSorting.sort(sortConfigs)
this.reSetCellValidationValues()
// Fix ARIA accessibility issues after table edit
setTimeout(() => {
this.fixAriaAccessibility()
}, 100)
}, 0)
}
convertToCorrectTypes(dataSource: any) {
for (const row of dataSource) {
for (const colKey in row) {
const colSpecs = this.cols.find((x: any) => x.NAME === colKey)
if (colSpecs) {
if (
row[colKey] !== '' &&
colSpecs.TYPE === 'num' &&
!colSpecs.DDTYPE.includes('TIME') &&
!colSpecs.DDTYPE.includes('DATE')
) {
const numericValue = parseFloat(row[colKey])
// A cell the file parser could not read as a number stays as it
// was: the editor's numeric validation then flags it as text,
// where a NaN would render as an empty cell.
if (!Number.isNaN(numericValue)) row[colKey] = numericValue
}
}
}
}
}
cancelEdit() {
this.cancelBulkValidation({ revert: false })
// Keep the context menu enabled after leaving edit mode (view mode keeps
// Copy/Export; editing items hide themselves when read-only).
this.toggleHotPlugin('contextMenu', true)
this.cellValidationSource = []
// Clear custom validation styling
this.clearDuplicateValidation()
const hot = this.hotInstance
const columnSorting = hot.getPlugin('multiColumnSorting')
const sortConfigs = this.getCurrentSortConfigs()
// Cleared here, before anything below reads a row by index - both the
// getFormulaCellsToPreserveOnCancel callbacks just below (physical
// rowIndex passed straight to visual-row-expecting Handsontable APIs)
// and the later updateSettings() call (see editTable()'s own comment -
// corrupts formula cell references while a sort is active) need the
// grid unsorted. Restored below, after the re-render, from the
// sortConfigs captured here.
if (sortConfigs.length > 0) columnSorting.clearSort()
if (this.dataSourceUnchanged) {
// dataSourceUnchanged deliberately holds the RAW pre-formula value for
// a HARDFORMULA/SOFTFORMULA column that still has its
// markOverwrittenCells comment (see editTable's overlay) - needed
// so classifyRow flags the row as modified, but it isn't a session
// edit to discard: the formula recomputes on every load regardless.
// Snapshot those cells' live computed value before the blind restore
// below, then patch them back in - otherwise cancelling would freeze
// the display at the raw value and erase the row's modified marker.
const formulaBaseCols = this.getFormulaBaseCols()
const commentsPlugin = hot.getPlugin('comments')
const toPreserve = getFormulaCellsToPreserveOnCancel(
this.dataSource.length,
formulaBaseCols,
(rowIndex, baseCol) =>
!!commentsPlugin.getCommentAtCell(
rowIndex,
hot.propToCol(baseCol) as number
),
(rowIndex, prop) => hot.getDataAtRowProp(rowIndex, prop)
)
this.dataSource = this.helperService.deepClone(this.dataSourceUnchanged)
for (const cell of toPreserve) {
if (this.dataSource[cell.rowIndex]) {
this.dataSource[cell.rowIndex][cell.prop] = cell.value
}
}
}
this.hotTable.data = this.dataSource
this.hotTable.readOnly = true
// Sort was already cleared above (before the getFormulaCellsToPreserveOnCancel
// callbacks); still cleared here going into updateSettings() - see
// editTable()'s own comment: it corrupts formula cell references
// ('#REF!') while a sort is active. Restored below via sortConfigs.
hot.updateSettings(
{
readOnly: this.hotTable.readOnly,
data: this.dataSource
},
false
)
// A cell's overwritten comment can be stale after this restore - e.g.
// a direct edit (not a formula overwrite) just got discarded, and
// dataSource now matches dataSourceRaw again for it. formula cells'
// comments are already correct (preserved above), so this is mostly
// about the general case, but running the same full resync either way
// is simpler than trying to only check what might have changed.
this.syncOverwrittenComments()
this.modifedRowsIndexes = []
hot.validateCells()
// this.editRecordListeners();
// sort() replaces the entire sort state on every call rather than
// accumulating (see its own JSDoc) - a multi-column sort needs one
// call with the whole array, not a loop of single-column calls, or
// every column but the last silently loses its sort.
if (sortConfigs.length > 0) columnSorting.sort(sortConfigs)
this.checkRowLimit()
// Back to read-only: apply any VA filter change that was held pending while
// editing (live mode). Confirm mode leaves it staged for the Apply button.
if (
this.isVaEmbed &&
this.vaAutoApply &&
this.vaFilterStatus === 'pending'
) {
this.applyPendingVaFilter()
}
}
/**
* Stop the bulk-validation flow (paste / autofill): invalidate in-flight
* responses via the epoch counter, reset spinner cells, hide the banner,
* and (when triggered from the banner's Cancel button) undo the change.
*/
public cancelBulkValidation(opts: { revert?: boolean } = {}) {
const wasActive = this.bulkValidation.active
// Invalidate any in-flight dynamicCellValidation responses. Note:
// sasService.request has no abort signal, so the network request itself
// keeps running — we only drop the response handling.
this.validationEpoch++
// Reset any cells still showing the loading spinner renderer.
const hot = this.hotInstance
if (hot && this.pendingSpinnerCells.size > 0) {
for (const key of this.pendingSpinnerCells) {
const [rStr, cStr] = key.split(',')
const r = Number(rStr)
const c = Number(cStr)
hot.setCellMeta(r, c, 'renderer', noSpinnerRenderer)
}
this.pendingSpinnerCells.clear()
}
// Drop placeholder entries (values still empty — request was cancelled).
this.cellValidationSource = this.cellValidationSource.filter(
(entry) => !entry.pending || entry.values.length > 0
)
this.bulkValidation = {
active: false,
done: 0,
total: 0
}
if (wasActive && opts.revert && hot) {
this.undoLastChange(hot)
}
if (hot) hot.render()
}
private undoLastChange(hot: Handsontable): void {
const plugin = hot.getPlugin('undoRedo') as unknown as
| { isUndoAvailable(): boolean; undo(): void }
| undefined
if (plugin?.isUndoAvailable()) plugin.undo()
}
/**
* Drive dynamic-source load + validation for cells that were bulk-filled
* (paste or autofill). HARDSELECT_HOOK / SOFTSELECT_HOOK columns need a SAS
* roundtrip; non-hook cells just need HOT's static validators. Caps backend
* concurrency, uses an epoch so a cancelled run can't mutate later state,
* and gates >3-cell runs behind a confirm modal.
*/
private async runBulkValidation(
hot: Handsontable,
ranges: Array<{
startRow: number
startCol: number
endRow: number
endCol: number
}>,
source: 'paste' | 'autofill'
): Promise<void> {
const rows = new Set<number>()
const hookTargets: Array<{ r: number; c: number }> = []
const hookCols = new Set<string>()
for (const range of ranges) {
for (let r = range.startRow; r <= range.endRow; r++) {
for (let c = range.startCol; c <= range.endCol; c++) {
rows.add(r)
const colKey = hot.colToProp(c) as string
if (this.dcValidator?.hasDqRules(colKey, ['HARDSELECT_HOOK'])) {
hookCols.add(colKey)
hookTargets.push({ r, c })
}
}
}
}
// No hook columns → HOT's own setDataAtCell → validateChanges pass
// (triggered by populateFromArray) already validates against the new
// value and paints htInvalid. A second validateRows here would race:
// it runs sync inside afterPaste/afterAutofill BEFORE applyChanges
// writes the data, captures the stale old value, and overwrites
// cellProperties.valid back to true — causing a 1-action lag.
if (hookTargets.length === 0) return
if (hookTargets.length === 1) {
const { r, c } = hookTargets[0]
await this.dynamicCellValidation(r, c)
hot.validateRows([...rows], () => hot.render())
return
}
if (hookTargets.length > 3) {
const colsList = [...hookCols].join(', ')
const ok = await this.showConfirmModal(
`Confirm ${source} validation`,
`You are about to trigger ${hookTargets.length} backend SAS request(s) for columns: ${colsList}. Do you wish to proceed?`
)
if (!ok) {
this.undoLastChange(hot)
return
}
}
const epoch = this.validationEpoch
this.bulkValidation = {
active: true,
done: 0,
total: hookTargets.length
}
const CONCURRENCY = 2
let idx = 0
await Promise.all(
Array.from({ length: CONCURRENCY }, async () => {
while (idx < hookTargets.length) {
if (epoch !== this.validationEpoch) return
const { r, c } = hookTargets[idx++]
await this.dynamicCellValidation(r, c, { skipRender: true })
if (epoch === this.validationEpoch) {
this.bulkValidation.done++
}
}
})
)
if (epoch !== this.validationEpoch) return
this.bulkValidation = {
...this.bulkValidation,
active: false
}
hot.validateRows([...rows], () => hot.render())
}
private showConfirmModal(title: string, message: string): Promise<boolean> {
this.confirmModal = { open: true, title, message }
return new Promise<boolean>((resolve) => {
this.confirmModalResolver = resolve
})
}
public onConfirmModalResult(value: boolean) {
const resolver = this.confirmModalResolver
this.confirmModalResolver = null
this.confirmModal = { ...this.confirmModal, open: false }
if (resolver) resolver(value)
}
timesClicked = 0
public hotClicked() {
if (this.timesClicked === 1 && this.hotTable.readOnly) {
this.editTable()
}
if (this.timesClicked === 0) {
this.timesClicked++
setTimeout(() => {
this.timesClicked = 0
}, 200)
}
}
public cleanExceed() {
this.exceedCells = []
}
public approversToggle() {
this.showApprovers = !this.showApprovers
}
public addRow() {
this.addingNewRow = true
setTimeout(() => {
const hot = this.hotInstance
const newIndex = this.dataSource.length
// hot.alter() (rather than splicing dataSource and calling
// updateSettings) is what triggers the formulas plugin's own
// insert-row hooks - without it, HyperFormula's sheet never learns
// about the new row and HARDFORMULA/SOFTFORMULA columns silently
// fall out of sync with the grid's own data.
hot.alter('insert_row_below', newIndex - 1, 1)
// alter() is expected to splice synchronously into dataSource, but
// this guards against relying on that if the data binding ever
// becomes async.
if (this.dataSource[newIndex]) {
this.dataSource[newIndex].noLinkOption = true
}
this.seedFormulaValuesForRow(newIndex)
this.updateEditStatusForRow(newIndex)
// Select the newly added row
hot.selectCell(newIndex, 0)
hot.render()
this.addingNewRow = false
this.reSetCellValidationValues()
})
}
/**
* Inserts a new row at the specified position and updates the table
*/
private insertRowAtPosition(targetRow: number): void {
const hot = this.hotInstance
// See addRow()'s comment - hot.alter() is required for HyperFormula to
// learn about the new row. beforeCreateRow only allows this while
// addingNewRow is set (see its own hook registration).
this.addingNewRow = true
hot.alter('insert_row_above', targetRow, 1)
this.addingNewRow = false
// alter() builds the new row from the grid's configured dataSchema
// (NOTNULL defaults included), which doesn't know about noLinkOption -
// set it directly on the row alter() just spliced into dataSource.
this.dataSource[targetRow].noLinkOption = true
this.seedFormulaValuesForRow(targetRow)
this.updateEditStatusForRow(targetRow)
this.reSetCellValidationValues()
}
/**
* Seeds HARDFORMULA/SOFTFORMULA columns on a newly-inserted row with
* their computed formula string, the same way applyFormulaRules seeds
* every row on initial load - a new row otherwise sits with a blank
* formula cell until the next full reload. DC.ORIG_VALUE naturally
* resolves to blank for this row (applyFormulaRules can't find a
* dataSourceUnchanged match for a row that never existed before).
*
* Written via hot.setDataAtRowProp (not a bare dataSource mutation, which
* is what applyFormulaRules itself does) so HyperFormula's engine
* actually learns about the new cell content - the same class of fix as
* hot.alter() above, just for a single cell instead of a row structure
* change.
*/
private seedFormulaValuesForRow(rowIndex: number): void {
const dqRules = this.dcValidator?.getDqDetails()
if (!dqRules || !hasFormulaRules(dqRules)) return
const hot = this.hotInstance
const userName = this.userService.user?.username ?? ''
const formulaRules = dqRules.filter(
(rule) =>
rule.RULE_TYPE === 'HARDFORMULA' || rule.RULE_TYPE === 'SOFTFORMULA'
)
// Computes only THIS row's formula string (parseFormulaRule directly,
// not applyFormulaRules on the full dataSource) - applyFormulaRules
// would overwrite every other row's formula column too, silently
// reverting any SOFTFORMULA override a user already typed elsewhere
// and rewriting every shifted row's cell-reference string (its
// spreadsheet position changed), both of which make classifyRow see a
// false diff against dataSourceUnchanged and misreport those untouched
// rows as modified.
for (const rule of formulaRules) {
const value = parseFormulaRule(rule.RULE_VALUE, {
columnNames: this.headerColumns,
rowIndex,
userName,
origValue: undefined
})
hot.setDataAtRowProp(rowIndex, rule.BASE_COL, value)
}
}
/**
* BASE_COL names of every HARDFORMULA/SOFTFORMULA rule whose result is
* expected to be *stable* for a given row (see getStableFormulaBaseCols
* for why DC.USER_NAME/DC.ORIG_VALUE/DC.ROW_STATUS-based rules are
* excluded). Used only by overlayFormulaRawValuesOnUnchanged and
* cancelEdit's preserve-on-cancel patch, which exist specifically to work
* around dataSourceUnchanged's formula-only raw overlay - see those
* methods' own doc comments for why that stays narrower than
* getRevertableColumnNames(). NOT the same filter seedFormulaValuesForRow
* uses above, which seeds every formula column regardless of this
* distinction.
*/
private getFormulaBaseCols(): string[] {
const dqRules = this.dcValidator?.getDqDetails()
if (!dqRules) return []
return getStableFormulaBaseCols(dqRules)
}
/**
* Overlays the true raw (pre-formula) value onto dataSourceUnchanged for
* every HARDFORMULA/SOFTFORMULA base col that already had real data (see
* markOverwrittenCells) - mutates in place. A HARDFORMULA/SOFTFORMULA
* rule can silently overwrite a value that already existed in the real
* dataset - that's a real change classifyRow's diff should pick up, not
* something that only becomes visible once an edit session starts. Shared
* by the initial load (so the row header agrees with EDIT_STATUS from the
* very first render) and editTable() (every time the baseline is rebuilt).
*/
private overlayFormulaRawValuesOnUnchanged(dataSourceUnchanged: any[]): void {
const formulaBaseCols = this.getFormulaBaseCols()
if (formulaBaseCols.length === 0 || !this.dataSourceRaw) return
dataSourceUnchanged.forEach((row, rowIndex) => {
const rawRow = this.dataSourceRaw[rowIndex]
if (!rawRow) return
for (const baseCol of formulaBaseCols) {
const rawValue = rawRow[baseCol]
if (rawValue === undefined || rawValue === null || rawValue === '')
continue
row[baseCol] = rawValue
}
})
}
/**
* Every column eligible to be checked/marked as "overwritten" - every
* revertable column (see getRevertableCols), not just the narrower
* formula-only set getFormulaBaseCols() returns. Used by
* markOverwrittenCells and the live afterChange sync - NOT by
* overlayFormulaRawValuesOnUnchanged/cancelEdit's preserve-on-cancel
* patch, which stay scoped to getFormulaBaseCols() specifically (see
* those methods' own doc comments for why).
*/
private getRevertableColumnNames(): string[] {
const dqRules = this.dcValidator?.getDqDetails()
if (!dqRules) return []
return getRevertableCols(dqRules, this.headerColumns)
}
/**
* Sets or clears a single cell's "overwritten" comment, matching its
* current value against dataSourceRaw (PK-matched, so this is safe to
* call after a row insert/delete/sort has moved things around). No-op if
* the row has no PK match in dataSourceRaw (a newly-inserted row) - see
* findOverwrittenCells for why that's never revertable.
*
* rowIndex is physical (matches dataSource's own order, same contract as
* syncOverwrittenComments()'s own dataSource.forEach caller below) -
* getDataAtRowProp and the comments plugin's own cell API all document a
* VISUAL row instead, so it's translated via hot.toVisualRow() for every
* Handsontable/plugin call, while dataSource itself stays indexed by the
* physical rowIndex throughout.
*/
private syncOverwrittenCommentForCell(rowIndex: number, prop: string): void {
const hot = this.hotInstance
const dataRow = this.dataSource[rowIndex]
if (!dataRow || !this.dataSourceRaw) return
const rawRow = this.dataSourceRaw.find((candidate) =>
this.headerPks.every((pk) => candidate[pk] === dataRow[pk])
)
if (!rawRow) return
const visualRow = hot.toVisualRow(rowIndex)
if (visualRow === null) return
const colIndex = hot.propToCol(prop) as number
const commentsPlugin = hot.getPlugin('comments')
const currentValue = hot.getDataAtRowProp(visualRow, prop)
const hasCommentAlready = !!commentsPlugin.getCommentAtCell(
visualRow,
colIndex
)
const action = syncOverwrittenCellComment(
currentValue,
rawRow[prop],
hasCommentAlready
)
if (action === 'set') {
commentsPlugin.setCommentAtCell(
visualRow,
colIndex,
`${EditorComponent.ORIGINAL_VALUE_COMMENT_PREFIX}${rawRow[prop]}`
)
} else if (action === 'remove') {
commentsPlugin.removeCommentAtCell(visualRow, colIndex)
}
}
/**
* Full resync of every revertable cell's "overwritten" comment against
* the current data. Used both at initial load (nothing has a comment
* yet, so this is purely additive) and after Cancel (an edit may have
* just been discarded, so a previously-set comment can now be stale) -
* Handsontable's comments plugin has no way to enumerate its own
* comments, so the only way to find a stale one is to re-check every
* candidate cell.
*/
private syncOverwrittenComments(): void {
const revertableCols = this.getRevertableColumnNames()
if (revertableCols.length === 0 || !this.dataSourceRaw) return
this.dataSource.forEach((_row, rowIndex) => {
for (const col of revertableCols) {
this.syncOverwrittenCommentForCell(rowIndex, col)
}
})
}
/**
* Marks every cell whose current value differs from what SAS actually
* sent for it (see findOverwrittenCells) with a read-only comment
* showing the original value - a HARDFORMULA/SOFTFORMULA rule silently
* overwriting real pre-existing data is one way this happens, but so is
* any direct edit; both need the same "here's what it used to be, and a
* way back" treatment. Must run after hot.updateSettings() has processed
* the formulas: getDataAtRowProp only resolves a formula cell's live
* computed value once HyperFormula has actually evaluated it, not the
* raw formula string dataSource holds right after applyFormulaRules.
*
* Also flips each affected row's EDIT_STATUS to 'M' - this is a real,
* permanent difference from the raw dataset (every future load
* recomputes the same formula again, and a direct edit made before this
* method ever ran isn't something afterChange would have seen), so
* nothing else would otherwise mark these rows modified. Doing this once
* here, before dataSourceUnchanged is ever cloned from dataSource, means
* the 'M' baseline is already shared by both snapshots - no special
* handling needed to keep it from being wiped out by a later
* cancelEdit().
*/
private markOverwrittenCells(): void {
this.syncOverwrittenComments()
const hot = this.hotInstance
const revertableCols = this.getRevertableColumnNames()
if (revertableCols.length === 0 || !this.dataSourceRaw) return
const currentRows = this.dataSource.map((row, rowIndex) => ({
...row,
...Object.fromEntries(
revertableCols.map((col) => [col, hot.getDataAtRowProp(rowIndex, col)])
)
}))
const changes = findOverwrittenCells(
currentRows,
this.dataSourceRaw,
revertableCols,
this.headerPks
)
const changedRows = new Set(changes.map((change) => change.rowIndex))
// Not classifyRow/updateEditStatusForRow - dataSourceUnchanged doesn't
// exist yet this early (only editTable() sets it), and these rows are
// already known-modified from the comment sync above; no need to
// reclassify.
//
// Deferred: setDataAtRowProp needs the Formulas plugin's hidden-column
// index mapping to have completed at least one settle pass. Called
// synchronously, right after the initial hot.updateSettings(), that
// mapping isn't ready yet and HyperFormula throws
// ExpectedValueOfTypeError (address.col resolves to undefined for the
// trimmed/hidden dc.row_status column) - since ngOnInit's getdata
// .catch() swallows anything thrown here, that leaves the whole table
// silently hidden. editTable() defers its own post-render work the
// same way, for the same index-mapping reason.
if (changedRows.size > 0) {
setTimeout(() => {
for (const rowIndex of changedRows) {
hot.setDataAtRowProp(
rowIndex,
EDIT_STATUS_COLUMN_NAME,
'M',
'editStatus'
)
}
}, 0)
}
}
/**
* A primary key identifies its own row for the rest of the edit session
* (getRowKey, dataModified, classifyRow all match rows by comparing PK
* values) - a HARDFORMULA/SOFTFORMULA rule on a PK column gets seeded by
* applyFormulaRules as a live `=...` formula string exactly like any
* other formula column, but a PK can never be allowed to keep
* recalculating for the rest of the session the way a normal formula
* column does: saveTable's own formulaRules loop only resolves a formula
* column's computed value once, at submit, so anything that changed a
* column the PK's formula depends on in between would silently shift the
* PK out from under dataModified/classifyRow's PK-matching - the same
* failure mode the afterChange hook below already prevents for a
* user-typed PK formula. So a PK-as-formula-column is frozen to its
* computed value once, right here at load. Both dataSource and
* dataSourceUnchanged get the same frozen value - overriding whatever
* overlayFormulaRawValuesOnUnchanged set for it, since a PK's identity
* role takes priority over the normal "did a formula overwrite real
* data" modified-detection every other formula column gets.
*/
private resolvePkFormulaSeedValues(): void {
const hot = this.hotInstance
const formulaBaseCols = getFormulaColumnNames(
this.dcValidator?.getDqDetails() ?? []
)
const pkFormulaCols = this.headerPks.filter((pk) =>
formulaBaseCols.includes(pk)
)
if (pkFormulaCols.length === 0) return
// Deferred for the same reason as this method's own caller
// (markOverwrittenCells) defers its setDataAtRowProp call above: right
// after the initial hot.updateSettings(), the Formulas plugin's
// hidden-column index mapping hasn't settled yet, and setDataAtRowProp
// throws ExpectedValueOfTypeError before it has.
setTimeout(() => {
this.dataSource.forEach((_row, rowIndex) => {
for (const pkCol of pkFormulaCols) {
const computed = hot.getDataAtRowProp(rowIndex, pkCol)
hot.setDataAtRowProp(rowIndex, pkCol, computed, 'resolvePkFormula')
const unchangedRow = this.dataSourceUnchanged?.[rowIndex]
if (unchangedRow) unchangedRow[pkCol] = computed
}
})
}, 0)
}
/**
* Recomputes this row's classification (M/A/D/U) and writes it into the
* EDIT_STATUS cell - the same classification the row-header symbol shows,
* but written into a real Handsontable cell so DC.ROW_STATUS-based
* formulas can actually react to it. Written via hot.setDataAtRowProp
* with a distinct source string (not a bare dataSource mutation) so
* HyperFormula is notified and dependents recalculate - the source string
* also lets the afterChange hook below recognise and ignore its own
* writes, avoiding infinite recursion.
*
* rowIndex is physical (matches dataSource's own order, same contract as
* addRow()/insertRowAtPosition()'s own callers) - setDataAtRowProp
* documents a VISUAL row instead, so it's translated via
* hot.toVisualRow() right before that one call, while dataSource itself
* stays indexed by the physical rowIndex.
*/
private updateEditStatusForRow(rowIndex: number): void {
const dataRow = this.dataSource[rowIndex]
if (!dataRow) return
const status = classifyRow(
dataRow,
this.dataSourceUnchanged ?? this.dataSource,
this.headerPks
)
const visualRow = this.hotInstance.toVisualRow(rowIndex)
if (visualRow === null) return
this.hotInstance.setDataAtRowProp(
visualRow,
EDIT_STATUS_COLUMN_NAME,
status,
'editStatus'
)
}
public cancelSubmit() {
this.dataSource = this.helperService.deepClone(this.dataSourceBeforeSubmit)
this.dataSourceBeforeSubmit = []
this.hotTable.data = this.dataSource
const hot = this.hotInstance
this.updateSettingsSortSafe(
{
data: this.dataSource,
colHeaders: getDisplayColHeaders(
this.headerColumns,
this.cols,
this.useLabels
),
columns: this.cellValidation,
modifyColWidth: function (width: number, col: number) {
if (col === 0) {
return 60
}
if (width > 500) return 500
else return width
}
},
false
)
hot.selectCell(0, 0)
hot.render()
hot.validateRows(this.modifedRowsIndexes)
this.reSetCellValidationValues()
}
public getRowsSubmittingCount() {
if (this.sasService.getSasjsConfig().debug) {
this.loggerService.log(this.dataSource)
this.loggerService.log(this.dataSourceUnchanged)
}
let rowsUpdated = 0
let rowsDeleted = 0
let rowsAdded = 0
this.modifedRowsIndexes = []
this.dataModified = []
for (let i = 0; i < this.dataSource.length; i++) {
const dataRow = this.helperService.deepClone(this.dataSource[i])
switch (classifyRow(dataRow, this.dataSourceUnchanged, this.headerPks)) {
case 'D':
this.dataModified.push(dataRow)
rowsDeleted++
break
case 'A':
this.dataModified.push(dataRow)
this.modifedRowsIndexes.push(i)
rowsAdded++
break
case 'M':
this.dataModified.push(dataRow)
this.modifedRowsIndexes.push(i)
rowsUpdated++
break
case 'U':
break
}
}
this.rowsChanged = {
rowsUpdated,
rowsDeleted,
rowsAdded
}
}
private clearDuplicateValidation() {
const hot = this.hotInstance
// Clear previous duplicate validation styling
for (const rowIndex of this.duplicatePkIndexes) {
for (let col = 1; col <= this.readOnlyFields; col++) {
hot.removeCellMeta(rowIndex, col, 'valid')
hot.removeCellMeta(rowIndex, col, 'dupKey')
// Remove our custom class from cell metadata
// getCellMeta<M>() defaults to Record<string, unknown> in HOT 18; pin it to
// CellMeta so `.className` keeps its real string | string[] type below.
const cellMeta = hot.getCellMeta<Handsontable.CellMeta>(rowIndex, col)
if (cellMeta.className) {
let cleanedClassName: string
if (Array.isArray(cellMeta.className)) {
cleanedClassName = cellMeta.className
.filter((c) => c !== 'dc-invalid-cell')
.join(' ')
} else {
cleanedClassName = cellMeta.className
.replace('dc-invalid-cell', '')
.trim()
}
hot.setCellMeta(rowIndex, col, 'className', cleanedClassName)
}
}
}
this.duplicatePkIndexes = []
hot.render()
}
public validatePrimaryKeys() {
const hot = this.hotInstance
// Clear previous validation before applying new ones
this.clearDuplicateValidation()
// Get data from the data source instead of hot.getData() to ensure consistency
const myTable = this.dataSource
this.pkFields = []
for (let index = 0; index < myTable.length; index++) {
let pkRow = ''
for (let ind = 1; ind < this.readOnlyFields + 1; ind++) {
const colName = this.headerColumns[ind]
const value = myTable[index][colName] || ''
pkRow = pkRow + '|' + value
}
this.pkFields.push(pkRow)
}
const results: any = []
const rows = this.dataSource.length
// Only check for duplicates if we have data
if (this.pkFields.length > 0) {
for (let j = 0; j < this.pkFields.length; j++) {
for (let i = 0; i < this.pkFields.length; i++) {
if (
this.pkFields[j] === this.pkFields[i] &&
i !== j &&
this.pkFields[j] !== '|'
) {
results.push(i)
}
}
}
}
// Clear any existing validation marks for all cells
for (let row = 0; row < myTable.length; row++) {
for (let col = 0; col < this.headerColumns.length; col++) {
const cellMeta = hot.getCellMeta(row, col)
if (cellMeta) {
cellMeta.valid = true
cellMeta.dupKey = false
}
}
}
// Mark duplicate cells as invalid
for (let k = 0; k < results.length; k++) {
for (let index = 1; index < this.readOnlyFields + 1; index++) {
hot.setCellMeta(results[k], index, 'valid', false)
hot.setCellMeta(results[k], index, 'dupKey', true)
hot.setCellMeta(results[k], index, 'className', 'dc-invalid-cell')
}
}
this.duplicatePkIndexes = [...new Set(results.sort())]
hot.render()
}
/**
* After any change or update to the hot datasource we lose cell validation values.
* This function is called in those places, to update all cells with values if existing.
* Note that was discussed:
* Rows with same data does not have arrows until you click on them (arrows gets lost after addRow)
* That is because this function resets the values for the found hashes, and if multiple rows have same data,
* hash table contains only first row that is hashed, so others don't get arrow re-set
*
* @param specificRowForceValue re-set will apply force values only to this row. That is used in cases
* when we don't want to re-set force values of every row in the table
*/
public reSetCellValidationValues(
setForcedValues = false,
specificRowForceValue?: number
) {
const hot = this.hotInstance
for (const entry of this.cellValidationSource) {
const colSource = entry.values.map(
(el: DynamicCellValidation) => el.RAW_VALUE
)
hot.batch(() => {
const cellMeta = hot.getCellMeta(entry.row, entry.col)
const cellRule = this.dcValidator?.getRule(
(cellMeta.data as string) || ''
)
let cellSource: string[] | number[] | undefined
if (cellRule) {
cellSource = this.dcValidator?.getDqDropdownSource(cellRule)
}
if (!cellSource) cellSource = []
const combinedSource = [
...new Set([...cellSource, ...colSource])
] as string[]
this.currentEditRecordValidator?.updateRule(entry.col, {
source: combinedSource
})
hot.setCellMeta(entry.row, entry.col, 'source', combinedSource)
if (entry.values.length > 0) {
hot.setCellMeta(entry.row, entry.col, 'renderer', 'autocomplete')
hot.setCellMeta(entry.row, entry.col, 'editor', 'autocomplete.custom')
hot.setCellMeta(entry.row, entry.col, 'strict', entry.strict)
hot.setCellMeta(entry.row, entry.col, 'filter', false)
this.currentEditRecordValidator?.updateRule(entry.col, {
renderer: 'autocomplete',
editor: 'autocomplete.custom',
strict: entry.strict,
filter: false
})
}
this.reSetExtendedCellValidationValues(
entry,
undefined,
setForcedValues,
specificRowForceValue
)
hot.render()
})
}
}
public reSetExtendedCellValidationValues(
cellValidationEntry?: CellValidationSource,
row?: number,
setForcedValues = false,
specificRowForceValue?: number
) {
const hot = this.hotInstance
if (cellValidationEntry) {
if (!row) row = cellValidationEntry.row
const extendedValuesObject =
this.getExtendedValuesByCellValue(cellValidationEntry)
this.setExtendedValuesToCells(
cellValidationEntry,
row,
extendedValuesObject,
setForcedValues,
specificRowForceValue
)
return
}
for (const entry of this.cellValidationSource) {
const extendedValuesObject = this.getExtendedValuesByCellValue(entry)
this.setExtendedValuesToCells(
entry,
entry.row,
extendedValuesObject,
setForcedValues,
specificRowForceValue
)
}
}
private setExtendedValuesToCells(
cellValidationEntry: CellValidationSource,
row: number,
extendedValues: DynamicExtendedCellValidation[],
setForcedValues = false,
specificRowForceValue?: number
) {
const hot = this.hotInstance
const uniqueCells: any[] = []
for (const element of extendedValues) {
if (uniqueCells.indexOf(element.EXTRA_COL_NAME) < 0)
uniqueCells.push(element.EXTRA_COL_NAME)
}
for (const cell of uniqueCells) {
const valuesForCol = extendedValues.filter(
(x) => x.EXTRA_COL_NAME === cell
)
let colSource: any = valuesForCol.map(
(el: DynamicExtendedCellValidation) =>
el.DISPLAY_TYPE === 'C' ? el.RAW_VALUE_CHAR : el.RAW_VALUE_NUM
)
const cellCol = hot.propToCol(cell) as number
const dynamicValidationEl =
this.disabledBasicDynamicCellValidationMap.find(
(x) => x.row === row && x.col === cellCol
)
if (!dynamicValidationEl) {
this.disabledBasicDynamicCellValidationMap.push({
row,
col: cellCol,
active: false
})
}
hot.setCellMeta(row, cellCol, 'renderer', 'autocomplete')
hot.setCellMeta(row, cellCol, 'editor', 'autocomplete.custom')
hot.setCellMeta(row, cellCol, 'strict', cellValidationEntry.strict)
hot.setCellMeta(row, cellCol, 'filter', false)
this.currentEditRecordValidator?.updateRule(cellCol, {
renderer: 'autocomplete',
editor: 'autocomplete.custom',
strict: cellValidationEntry.strict,
filter: false
})
const cellMeta = hot.getCellMeta(row, cellCol)
const cellRule = this.dcValidator?.getRule(
(cellMeta.data as string) || ''
)
let cellSource: string[] | number[] | undefined
if (cellRule) {
cellSource = this.dcValidator?.getDqDropdownSource(cellRule)
}
if (!cellSource) cellSource = []
if (cellRule?.type === 'numeric') {
cellSource = this.helperService.convertArrayValues(
cellSource,
'number'
) as number[]
colSource = this.helperService.convertArrayValues(
colSource,
'number'
) as number[]
} else {
cellSource = this.helperService.convertArrayValues(
cellSource,
'string'
) as string[]
colSource = this.helperService.convertArrayValues(
colSource,
'string'
) as string[]
}
const combinedSource = [...new Set([...cellSource, ...colSource])]
hot.setCellMeta(row, cellCol, 'source', combinedSource)
this.currentEditRecordValidator?.updateRule(cellCol, {
source: combinedSource
})
if (setForcedValues) {
if (specificRowForceValue && specificRowForceValue !== row) {
return
}
const forceValue = valuesForCol.find(
(x: DynamicExtendedCellValidation) => x.FORCED_VALUE === 1
)
if (forceValue) {
// Adding timeout here makes forced values cell to re-validate itself
setTimeout(() => {
hot.setDataAtCell(
row,
cellCol,
forceValue.DISPLAY_TYPE === 'C'
? forceValue.RAW_VALUE_CHAR
: forceValue.RAW_VALUE_NUM,
'force_cell_validation_value'
)
if (this.currentEditRecordIndex === row) {
this.dataSource[this.currentEditRecordIndex][cell] =
forceValue.DISPLAY_TYPE === 'C'
? forceValue.RAW_VALUE_CHAR
: forceValue.RAW_VALUE_NUM
}
})
}
}
}
}
/**
* Parses values of extended cell validation for the given dynamic cell validation entry
* @param cellValidationEntry stored dynamic cell validation entry from which to parse extended validation values
* @param rowOverride if not provided, row that is used is row found in `cellValidationEntry`. This is needed when for example we change the w in hot
* we need to get `cellValue` from that row and not from hashed cell validation source.
* @returns extended values object
*/
private getExtendedValuesByCellValue(
cellValidationEntry: CellValidationSource,
rowOverride?: number
): DynamicExtendedCellValidation[] {
const hot = this.hotInstance
const cellValue = hot.getDataAtCell(
rowOverride ? rowOverride : cellValidationEntry.row,
cellValidationEntry.col
)
const valueIndex = (
cellValidationEntry.values.find((x) => x.RAW_VALUE === cellValue) || {}
).DISPLAY_INDEX
return (
cellValidationEntry.extended_values?.filter(
(x) => x.DISPLAY_INDEX === valueIndex
) || []
)
}
public checkSave() {
this.getRowsSubmittingCount()
if (
this.rowsChanged.rowsAdded === 0 &&
this.rowsChanged.rowsUpdated === 0 &&
this.rowsChanged.rowsDeleted === 0
) {
this.badEditTitle = 'No changes to submit'
this.badEditCause = 'Please modify some values and try again.'
this.badEdit = true
return
}
const hot = this.hotInstance
this.dataSourceBeforeSubmit = this.helperService.deepClone(this.dataSource)
// Clean up the data source by removing noLinkOption property
for (let i = 0; i < this.dataSource.length; i++) {
delete this.dataSource[i].noLinkOption
}
// Remove any completely empty rows from the end
while (this.dataSource.length > 0) {
const lastRow = this.dataSource[this.dataSource.length - 1]
const isEmpty = Object.keys(lastRow).every((key) => {
if (
key === '_____DELETE__THIS__RECORD_____' ||
key === EDIT_STATUS_COLUMN_NAME
)
return true
return !lastRow[key] || lastRow[key] === ''
})
if (isEmpty) {
this.dataSource.pop()
} else {
break
}
}
this.updateSettingsSortSafe(
{
data: this.dataSource,
colHeaders: getDisplayColHeaders(
this.headerColumns,
this.cols,
this.useLabels
),
columns: this.cellValidation,
modifyColWidth: function (width: number, col: number) {
if (width > 500) return 500
else return width
}
},
false
)
this.reSetCellValidationValues()
EditorComponent.cnt = 0
EditorComponent.nonPkCnt = 0
this.validatePrimaryKeys()
if (this.duplicatePkIndexes.length !== 0) {
this.pkDups = true
this.submit = false
this.cancelSubmit()
return
} else {
this.pkDups = false
}
hot.validateRows(this.modifedRowsIndexes, () => {
if (this.checkInvalid()) {
const abortMsg = 'Invalid Values are Present'
this.eventService.showInfoModal('Validation error', abortMsg)
return
}
this.submit = true
this.validationDone = 1
setTimeout(() => {
const txt: any = document.getElementById('formFields_8')
if (txt) txt.focus()
}, 200)
})
}
public async saveTable(data: any) {
const hot = this.hotInstance
// HARDFORMULA/SOFTFORMULA columns hold a live '=...' formula string in
// dataSource - HyperFormula recalculates the DISPLAYED value without
// ever mutating that underlying string, so submitting dataSource as-is
// would send the formula text itself instead of its computed result.
// getDataAtRowProp reads through the formulas plugin's own modifyData
// hook, which resolves the engine's live value for that cell. Must run
// over the full, unfiltered `data` (physical/array order, same
// reference as dataSource) - its index is what toVisualRow/
// getDataAtRowProp need; the filtered/mapped array below no longer
// lines up with actual grid rows.
const formulaRules = (this.dcValidator?.getDqDetails() ?? []).filter(
(rule) =>
rule.RULE_TYPE === 'HARDFORMULA' || rule.RULE_TYPE === 'SOFTFORMULA'
)
if (formulaRules.length > 0) {
data.forEach((row: any, physicalRowIndex: number) => {
const visualRowIndex = hot.toVisualRow(physicalRowIndex)
for (const rule of formulaRules) {
row[rule.BASE_COL] = hot.getDataAtRowProp(
visualRowIndex,
rule.BASE_COL
)
}
})
}
// Two things every other column (HARDFORMULA/SOFTFORMULA ones are
// already fully handled above) still needs before submission:
//
// 1. Strip the escape marker this app added (see
// escapeCharacterColumnValue/character-column-formula-plan.md) from
// every cell the app itself marked, so it never reaches the
// backend. A cell the user has since edited, or one whose leading
// `'` was already genuine backend data, is never in
// autoEscapedCells and so is never touched here - it submits
// exactly as it currently reads. autoEscapedCells is only ever
// populated for character columns, so this is naturally a no-op
// for any other column.
// 2. Resolve the computed value of a genuine, currently-live formula
// (typed, pasted, or "Apply as formula"'d - never auto-escaped, so
// its raw dataSource string still starts with a bare `=`) the same
// way the HARDFORMULA/SOFTFORMULA loop above does - otherwise the
// raw formula text itself would reach the backend instead of what
// the grid actually displays. Not scoped to character columns -
// HyperFormula evaluates a `=`-led value the same way regardless
// of the column's own declared type, and coerceNumericRow
// (beforePaste/beforeAutofill) deliberately leaves a non-numeric,
// formula-looking paste alone rather than forcing it to NaN, so a
// numeric column can end up holding a live formula too.
//
// Runs regardless of whether this table has any HARDFORMULA/
// SOFTFORMULA rules - formulas (and both of the above) are enabled
// everywhere.
const formulaBaseCols = getFormulaColumnNames(
this.dcValidator?.getDqDetails() ?? []
)
data.forEach((row: any, physicalRowIndex: number) => {
const visualRowIndex = hot.toVisualRow(physicalRowIndex)
const rowKey = getRowKey(row, this.headerPks)
for (const colName of this.headerColumns) {
if (formulaBaseCols.includes(colName)) continue
if (isAutoEscaped(this.autoEscapedCells, rowKey, colName)) {
row[colName] = unescapeFormula(row[colName])
} else if (
typeof row[colName] === 'string' &&
row[colName].startsWith('=')
) {
row[colName] = hot.getDataAtRowProp(visualRowIndex, colName)
}
}
})
data = data.filter((dataRow: any) => {
const elModified = this.dataModified.find((row) => {
for (const pkCol of this.headerPks) {
if (row[pkCol] !== dataRow[pkCol]) {
return false
}
}
return true
})
return !!elModified
})
data = data.map((row: any) => {
const deleteColValue = row['_____DELETE__THIS__RECORD_____']
delete row['_____DELETE__THIS__RECORD_____']
row['_____DELETE__THIS__RECORD_____'] = deleteColValue
// EDIT_STATUS is client-only (see editStatusColumnRule.ts) - it must
// never reach the backend.
delete row[EDIT_STATUS_COLUMN_NAME]
// If cell is numeric and value is dot `.` we change it to `null`
Object.keys(row).map((key: string) => {
const colRule = this.dcValidator?.getRule(key)
if (colRule?.type === 'numeric' && row[key] === '.') row[key] = null
})
return row
})
this.loggerService.log('Data submitted', data)
if (this.checkInvalid()) {
const abortMsg = 'Invalid Values are Present'
this.eventService.showInfoModal('Validation error', abortMsg)
this.cancelSubmit()
this.submit = false
return
}
this.validationDone = 0
this.saveLoading = true
if (
EditorComponent.cnt < 1 &&
this.duplicatePkIndexes.length === 0 &&
EditorComponent.nonPkCnt < 1
) {
this.saveLoading = true
this.disableSubmit = false
this.submit = true
const updateParams: any = {}
updateParams.ACTION = 'LOAD'
this.message = sanitiseForSas(this.message.replace(/\n/g, '. '))
updateParams.MESSAGE = this.message
// updateParams.APPROVER = this.approver;
updateParams.LIBDS = this.libds
if (this.cols) {
const submitData = data.slice(
0,
this.licenceState.value.submit_rows_limit
)
const success = await this.sasStoreService
.updateTable(
updateParams,
submitData,
'SASControlTable',
'editors/stagedata',
this.$dataFormats
)
.then((res: RequestWrapperResponse) => {
if (typeof res.adapterResponse.sasparams !== 'undefined') {
this.router.navigateByUrl(
'/stage/' + res.adapterResponse.sasparams[0].DSID
)
return true
}
let error = `Submit request failed`
if (res) {
const errorText =
typeof res === 'string' ? res : JSON.stringify(res)
error += `\n${errorText}`
}
this.eventService.showAbortModal(
'editors/stagedata',
error,
null,
'Submit error'
)
})
.catch((err: any) => {
console.log('err', err)
EditorComponent.cnt = 0
EditorComponent.nonPkCnt = 0
this.disableSubmit = true
this.submit = false
/* sasService has already shown the SAS message when it rejected
(the abort, or the adapter's error), so reporting it again here
stacked a second modal on top of the first - and read
'undefined' because it looked for a property that does not
exist. Only speak up when nothing was shown. */
if (!err?.errorShown) {
const errorText =
typeof err?.adapterResponse?.error === 'string'
? err.adapterResponse.error
: JSON.stringify(err?.adapterResponse ?? err)
this.eventService.showAbortModal(
'editors/stagedata',
`Submit request failed\n${errorText}`,
null,
'Submit error'
)
}
return false
})
if (success) return //stop code execution if route redirected
}
}
if (EditorComponent.cnt >= 1) {
this.pkNull = true
this.submit = true
} else {
this.submit = false
}
if (EditorComponent.nonPkCnt >= 1) {
this.noPkNull = true
this.submit = true
} else {
this.submit = false
}
this.cancelSubmit()
EditorComponent.cnt = 0
EditorComponent.nonPkCnt = 0
this.disableSubmit = true
}
public validatorRuleSource(colName: string) {
return this.dcValidator?.getRule(colName)
}
public checkInvalid() {
// Use Angular wrapper to access Handsontable element instead of DOM queries
if (!this.hotTableComponent || !this.hotTableComponent.hotInstance)
return false
const hotElement = this.hotTableComponent.hotInstance.rootElement
if (!hotElement) return false
// Check for standard Handsontable validation failures
const standardInvalidCells = hotElement.querySelectorAll('.htInvalid')
// Check for our custom duplicate primary key validation failures
const customInvalidCells = hotElement.querySelectorAll('.dc-invalid-cell')
return standardInvalidCells.length > 0 || customInvalidCells.length > 0
}
public goToEditor() {
this.router.navigateByUrl('/')
}
closeRecordEdit(confirmButtonClicked?: boolean) {
this.currentEditRecordIndex = -1
this.currentEditRecord = undefined
this.currentEditRecordValidator = undefined
if (this.recordAction === 'ADD' && !confirmButtonClicked) {
this.dataSource = this.helperService.deepClone(this.prevDataSource)
this.updateSettingsSortSafe(
{
data: this.dataSource
},
false
)
}
}
confirmRecordEdit(close = true) {
const closingRecordIndex = this.currentEditRecordIndex
if (close) this.currentEditRecordIndex = -1
this.columnHeader.map((colName: string) => {
const value = this.currentEditRecord[colName]
const isNum = this.$dataFormats?.vars[colName]?.type === 'num'
const specialMissing = isSpecialMissing(value)
if (isNum && !isNaN(value) && !specialMissing) {
this.currentEditRecord[colName] = value * 1
}
})
this.dataSource[closingRecordIndex] = this.currentEditRecord
this.hotTable.data[closingRecordIndex] = this.currentEditRecord
this.updateSettingsSortSafe(
{
data: this.dataSource
},
false
)
if (close) this.currentEditRecord = undefined
}
onNextRecord() {
this.confirmRecordEdit(false)
this.currentEditRecordIndex =
this.currentEditRecordIndex >= this.dataSource.length - 1
? 0
: this.currentEditRecordIndex + 1
this.editRecord(null, this.currentEditRecordIndex)
}
onPreviousRecord() {
this.confirmRecordEdit(false)
this.currentEditRecordIndex =
this.currentEditRecordIndex <= 0
? this.dataSource.length - 1
: this.currentEditRecordIndex - 1
this.editRecord(null, this.currentEditRecordIndex)
}
addRecordButtonClick() {
if (this.restrictions.restrictAddRecord) {
this.eventService.showDemoLimitModal('Add Record')
return
}
this.addEditNewRecord()
}
addEditNewRecord() {
this.addRecord()
setTimeout(() => {
this.editRecord(null, this.dataSource.length - 1, true)
}, 1000)
}
addRecord() {
this.addRow()
}
editRecord(item: Element | null, index?: number, newRecord?: boolean) {
if (this.restrictions.restrictEditRecord) {
this.eventService.showDemoLimitModal('Edit Record')
return
}
if (index === undefined || index < 0) return
if (this.restrictions.restrictEditRecord) {
return
}
this.recordAction = newRecord ? 'ADD' : 'EDIT'
if (this.hotTable.readOnly) {
this.editTable(false, newRecord)
}
// Create copy of DC validator to be used in RECORD MODAL
this.currentEditRecordValidator = this.helperService.deepClone(
this.dcValidator
)
if (newRecord) {
this.prevDataSource = this.helperService.deepClone(this.dataSource)
this.prevDataSource.pop()
} else {
const currentEditRecordCellsMeta = this.helperService.deepClone(
this.hotInstance.getCellMetaAtRow(
index
) as Partial<Handsontable.CellProperties>[]
)
// Update that copy with current cells meta (dynamic validation data)
for (const cellMeta of currentEditRecordCellsMeta) {
if (cellMeta) {
const data = cellMeta.prop?.toString() //------------
delete cellMeta.prop // We convert to be able to update dcValidator rule by using CellProperties
delete cellMeta.data //-----------------
this.currentEditRecordValidator?.updateRule(cellMeta.col!, {
...cellMeta,
data: data
})
}
}
}
this.currentEditRecordIndex = index
this.currentEditRecord = this.helperService.deepClone(
this.dataSource[index]
)
}
toggleHotPlugin(pluginName: string, enable: boolean) {
const hot = this.hotInstance
const contextMenuPlugin = hot.getPlugin<any>(pluginName)
if (!contextMenuPlugin) {
console.warn(
'Toggle Hot Plugin failed - Plugin named: ' +
pluginName +
' - could not be found.'
)
return
}
setTimeout(() => {
// The instance may be destroyed/rebuilt within this 100ms window (e.g. a
// VA filter reload repopulates the grid). A destroyed plugin has its `hot`
// reference deleted, so enablePlugin()/disablePlugin() would throw on
// `this.hot.getSettings()`. Bail if the instance is gone or was swapped.
if (
hot.isDestroyed ||
this.hotInstance !== hot ||
!contextMenuPlugin.hot
) {
return
}
if (enable) {
contextMenuPlugin.enablePlugin()
} else {
contextMenuPlugin.disablePlugin()
}
hot.render()
}, 100)
}
private dynamicCellValidationDisabled(row: number, col: number) {
const rowColFound = this.disabledBasicDynamicCellValidationMap.find(
(x) => x.row === row && x.col === col && !x.active
)
return !!rowColFound
}
/**
* This function takes row and column numbers for the cel to be validated and pouplated with values.
* It will send the row values without the current column to the sas.
* Sas will return values and if length greater then zero cel becomes dropdown type and values from sas
* put to source.
* @param row handsontable row
* @param column handsontable column
*/
public async dynamicCellValidation(
row: number,
column: number,
opts?: { skipRender?: boolean },
retried = false
): Promise<void> {
if (this.dynamicCellValidationDisabled(row, column))
return Promise.resolve()
const hot = this.hotInstance
const cellMeta = hot.getCellMeta(row, column)
if (cellMeta.readOnly) return Promise.resolve()
const cellData = hot.getDataAtCell(row, column)
const clickedRow = this.helperService.deepClone(this.dataSource[row])
// hot.colToProp resolves the column's actual configured data key -
// Object.keys(clickedRow)[column] only worked by coincidence for a
// fully-populated row; a sparse row (one whose backend source never
// populated some earlier column) has fewer real keys than the grid has
// columns, so counting into its own key list drifts out of alignment
// (or runs out of bounds entirely) once column indexes past the gap.
const clickedColumnKey = hot.colToProp(column) as string
const skipRender = !!opts?.skipRender
const myEpoch = this.validationEpoch
/**
* We will hash the row (without current column) so later we check if hash is the same
* we set the values relative to that hash
* if not we fire the request.
*/
const hashedRow = this.helperService.deleteKeysAndHash(
clickedRow,
[clickedColumnKey, 'noLinkOption'],
false
)
const validationSourceIndex = this.cellValidationSource.findIndex(
(entry: CellValidationSource) => entry.hash === hashedRow
)
/**
* Set the values for found hash.
*/
if (validationSourceIndex > -1) {
// In-flight dedup: another call with the same hash is mid-request.
// Wait for it then re-enter once so we walk the populated cache-hit
// path instead of validating against the empty placeholder.
const inFlight = this.cellValidationSource[validationSourceIndex].pending
if (inFlight && !retried) {
try {
await inFlight
} catch {
/* swallowed — original caller handles */
}
if (myEpoch !== this.validationEpoch) return
return this.dynamicCellValidation(row, column, opts, true)
}
}
if (validationSourceIndex > -1) {
let colSource = this.cellValidationSource[
validationSourceIndex
].values.map((el) => el.RAW_VALUE)
// `.source` (dropdown/autocomplete list) is typed unknown[] | Function here since
// getCellMeta() isn't given a type param; cast to the array branch we actually use.
const cellHadSource =
((hot.getCellMeta(row, column).source as unknown[]) || []).length < 1
const cellHasValue = cellData !== ' '
hot.batch(() => {
const cellMeta = hot.getCellMeta(row, column)
const cellRule = this.dcValidator?.getRule(
(cellMeta.data as string) || ''
)
let cellSource: string[] | number[] | undefined
if (cellRule) {
cellSource = this.dcValidator?.getDqDropdownSource(cellRule)
}
if (!cellSource) cellSource = []
if (cellRule?.type === 'numeric') {
cellSource = this.helperService.convertArrayValues(
cellSource,
'number'
) as number[]
colSource = this.helperService.convertArrayValues(
colSource,
'number'
) as number[]
} else {
cellSource = this.helperService.convertArrayValues(
cellSource,
'string'
) as string[]
colSource = this.helperService.convertArrayValues(
colSource,
'string'
) as string[]
}
const cellSourceCombined = [
...new Set([...cellSource, ...colSource])
] as string[]
hot.setCellMeta(row, column, 'source', cellSourceCombined)
this.currentEditRecordValidator?.updateRule(column, {
source: cellSourceCombined
})
if (
this.cellValidationSource[validationSourceIndex].values.length > 0
) {
const strict = this.cellValidationSource[validationSourceIndex].strict
hot.setCellMeta(row, column, 'renderer', 'autocomplete')
hot.setCellMeta(row, column, 'editor', 'autocomplete.custom')
hot.setCellMeta(row, column, 'strict', strict)
hot.setCellMeta(row, column, 'filter', false)
this.currentEditRecordValidator?.updateRule(column, {
renderer: 'autocomplete',
editor: 'autocomplete.custom',
strict: strict,
filter: false
})
}
this.reSetExtendedCellValidationValues(
this.cellValidationSource[validationSourceIndex],
row,
cellHadSource && cellHasValue
)
if (!skipRender) hot.render()
})
} else if (validationSourceIndex < 0) {
/**
* Send request to sas.
*/
const data = {
SASControlTable: [
{
libds: this.libds,
variable_nm: clickedColumnKey
}
],
// EDIT_STATUS is client-only (see editStatusColumnRule.ts) - it
// must never reach the backend.
source_row: [withoutEditStatus(clickedRow)]
}
const validationHook = this.dcValidator
?.getDqDetails(clickedColumnKey)
.find(
(rule: DQRule) =>
rule.RULE_TYPE === 'SOFTSELECT_HOOK' ||
rule.RULE_TYPE === 'HARDSELECT_HOOK'
)
/**
* Do the validation only if current column has validation hooks in place.
*/
if (validationHook) {
this.cellValidationSource.push({
row: row,
col: column,
strict: validationHook.RULE_TYPE === 'HARDSELECT_HOOK',
values: [],
hash: hashedRow,
count: this.cellValidationSource.length + 1
})
this.currentEditRecordLoadings.push(column)
const spinnerKey = `${row},${column}`
// Defer the spinner renderer so the click event finishes settling
// HOT's focus catcher before we replace td.innerHTML. Without this
// defer, clicking the cell loses focus immediately.
// Skip the spinner entirely if SAS responds before this fires —
// avoids the brief flicker users were seeing on fast responses.
// Also skip during paste (skipRender) — the progress banner handles
// visual feedback and cell spinners would just flicker.
const spinnerTimeout: ReturnType<typeof setTimeout> | null = skipRender
? null
: setTimeout(() => {
hot.setCellMeta(row, column, 'renderer', spinnerRenderer)
this.pendingSpinnerCells.add(spinnerKey)
hot.render()
}, 150)
const pendingPromise = this.sasService
.request('editors/getdynamiccolvals', data, undefined, {
suppressSuccessAbortModal: true,
suppressErrorAbortModal: true
})
.then(async (res: RequestWrapperResponse) => {
if (spinnerTimeout) clearTimeout(spinnerTimeout)
this.pendingSpinnerCells.delete(spinnerKey)
// Cancelled mid-flight — drop the placeholder entry so future
// calls don't see an empty cache hit, and skip all UI work.
if (myEpoch !== this.validationEpoch) {
const idx = this.cellValidationSource.findIndex(
(e) => e.hash === hashedRow
)
if (idx > -1) this.cellValidationSource.splice(idx, 1)
return
}
const colSource = res.adapterResponse.dynamic_values.map(
(el: DynamicCellValidation) => el.RAW_VALUE
)
this.currentEditRecordLoadings.splice(
this.currentEditRecordLoadings.indexOf(column),
1
)
if (colSource.length > 0) {
const validationSourceIndex = this.cellValidationSource.findIndex(
(entry: CellValidationSource) => entry.hash === hashedRow
)
if (validationSourceIndex > -1) {
this.cellValidationSource[validationSourceIndex] = {
...this.cellValidationSource[validationSourceIndex],
row: row,
col: column,
values: res.adapterResponse.dynamic_values,
extended_values: res.adapterResponse.dynamic_extended_values,
pending: undefined
}
}
/**
* In the case that the original value is not included in the newly created cell dropdown
* and validation type is HARDSELECT, the cell shoud be red
*/
await new Promise<void>((resolve) =>
setTimeout(() => {
this.reSetCellValidationValues(true, row)
if (!skipRender) {
hot.render()
hot.validateRows([row])
}
resolve()
}, 100)
)
} else {
if (!skipRender) {
hot.setCellMeta(row, column, 'renderer', noSpinnerRenderer)
hot.render()
}
const idx = this.cellValidationSource.findIndex(
(e) => e.hash === hashedRow
)
if (idx > -1) this.cellValidationSource[idx].pending = undefined
}
/**
* If hash table limit reached, remove the oldest element.
* Oldest element is element with lowest `count` number.
*/
if (this.cellValidationSource.length > this.validationTableLimit) {
const oldestElement = this.cellValidationSource.reduce(
(prev, curr) => (prev.count < curr.count ? prev : curr)
)
const oldestElementIndex =
this.cellValidationSource.indexOf(oldestElement)
this.cellValidationSource.splice(oldestElementIndex, 1)
}
})
.catch((err: any) => {
if (spinnerTimeout) clearTimeout(spinnerTimeout)
this.pendingSpinnerCells.delete(spinnerKey)
const currentRowHashIndex = this.cellValidationSource.findIndex(
(x) => x.hash === hashedRow
)
this.cellValidationSource.splice(currentRowHashIndex, 1)
if (myEpoch !== this.validationEpoch) return
if (!skipRender) {
hot.batch(() => {
// Render error icon inside a cell
hot.setCellMeta(row, column, 'renderer', errorRenderer)
hot.render()
})
}
//Stop edit record modal loading spinner
this.currentEditRecordLoadings.splice(
this.currentEditRecordLoadings.indexOf(column),
1
)
//Show error on edit record modal
this.currentEditRecordErrors.push(column)
// After waiting time remove the error icon from cell and edit record modal field
setTimeout(() => {
if (!skipRender) {
hot.setCellMeta(row, column, 'renderer', noSpinnerRenderer)
hot.render()
}
//Remove error icon on the edit record modal field
this.currentEditRecordErrors.splice(
this.currentEditRecordErrors.indexOf(column),
1
)
}, 3000)
this.reSetCellValidationValues()
this.loggerService.log('getdynamiccolvals error:', err)
})
const entryIdx = this.cellValidationSource.findIndex(
(e) => e.hash === hashedRow
)
if (entryIdx > -1) {
this.cellValidationSource[entryIdx].pending = pendingPromise
}
return pendingPromise
}
}
return Promise.resolve()
}
checkEmptyRowWhenFilter() {
this.zeroFilterRows = false
if (
typeof this.filter_pk !== 'undefined' &&
this.hotTable.data.length === 1
) {
if ([null, ''].includes(this.hotTable.data[0][this.headerPks[0]]))
this.zeroFilterRows = true
}
}
onRecordInputFocus(event: EditRecordInputFocusedEvent) {
this.dynamicCellValidation(this.currentEditRecordIndex, event.colName)
}
executeDynamicCellValidationIfApplicable(colProp: any, col: any, row: any) {
const hashedRow = this.helperService.deleteKeysAndHash(
this.dataSource[row],
[colProp, 'noLinkOption']
)
const cellValidation = this.cellValidationSource.find(
(entry: CellValidationSource) =>
entry.hash === hashedRow && col === entry.col
)
if (
cellValidation &&
cellValidation.extended_values &&
cellValidation.extended_values.length > 0
) {
const extendedValidationObject = this.getExtendedValuesByCellValue(
cellValidation,
row
)
this.setExtendedValuesToCells(
cellValidation,
row,
extendedValidationObject,
true
)
}
}
datasetInfoModalRowClicked(value: Version | DSMeta) {
if ((<Version>value).LOAD_REF !== undefined) {
// Type is Version
const row = value as Version
const url = `/stage/${row.LOAD_REF}`
this.router.navigate([url])
}
}
viewboxManager() {
this.viewboxes = true
}
get totalRowsChanged() {
return (
this.rowsChanged.rowsUpdated +
this.rowsChanged.rowsDeleted +
this.rowsChanged.rowsAdded
)
}
/**
* Function checks if selected hot cell is solo cell selected
* and if it is, set the `filter` property based on filter param.
*
* @param filter
*/
private setCellFilter(filter: boolean) {
const hotSelected = this.hotInstance.getSelected()
if (!hotSelected) return
const selection = hotSelected ? hotSelected[0] : hotSelected
// When we open a dropdown we want filter disabled so value in cell
// don't filter out items, since we want to see them all.
// But when we start typing we want to be able to start filtering values
// again
if (selection) {
const startRow = selection[0]
const endRow = selection[2]
const startCell = selection[1]
const endCell = selection[3]
if (startRow === endRow && startCell === endCell) {
const cellMeta = this.hotInstance.getCellMeta(startRow, startCell)
// If filter is not already set at the value in the param, set it
if (cellMeta && cellMeta.filter === !filter) {
this.hotInstance.setCellMeta(startRow, startCell, 'filter', filter)
}
}
}
}
async ngOnInit() {
// VA data-driven content: register the postMessage listener as early as
// possible (like the SAS sample's load-time registration) so VA's initial
// data message isn't missed during the editor's getdata load. The message
// is buffered on the service and replayed once the grid is ready.
if (this.isVaEmbed) this.subscribeToVaMessages()
// Initialize hot table settings
this.updateHotTableSettings()
this.licenceService.hot_license_key.subscribe(
(hot_license_key: string | undefined) => {
this.hotTable.licenseKey = hot_license_key
this.updateHotTableSettings() // Update settings when license key changes
}
)
// URL is the source of truth for the labels toggle: react to `?labels=`
// changes (including same-page navigation from the dropdown menu item)
// rather than reading it once from the route snapshot.
this._queryParams = this.route.queryParams.subscribe((params) => {
this.useLabels = parseLabelsParam(new URLSearchParams(params).toString())
this.applyDisplayColHeaders()
})
this._query = this.sasStoreService.query.subscribe((query: any) => {
if (query.libds === this.libds) {
this.whereString = query.string
this.clauses = query.obj
// this.libds = query.libds
}
})
// recover lib and table parameters from url; filter pk is optional - if filter is applied
const myParams: any = {}
if (typeof this.route.snapshot.params['libMem'] !== 'undefined') {
this.libds = this.route.snapshot.params['libMem']
this.filter_pk = this.route.snapshot.params['filterId']
if (this.route.snapshot.url[0].path === 'edit-record') {
if (typeof this.filter_pk !== 'undefined') {
this.recordAction = 'EDIT'
} else {
this.recordAction = 'ADD'
}
}
myParams.LIBDS = this.libds
if (typeof this.filter_pk !== 'undefined') {
myParams.FILTER_RK = parseInt(this.filter_pk)
}
myParams.OUTDEST = 'WEB'
if (this.libds) {
globals.editor.library = this.libds.split('.')[0]
globals.editor.table = this.libds.split('.')[1]
}
}
// VA initial load: if VA's first message already carries a filter, apply it
// BEFORE the first getdata so the initial fetch is filtered (no full-table
// load + reload). Navigates away on success; this instance aborts its load.
// Falls back to the normal unfiltered load on timeout/no-filter/failure.
if (await this.maybeDeferInitialVaLoad()) return
if (this.libds) {
this.getdataError = false
await this.sasStoreService
.callService(myParams, 'SASControlTable', 'editors/getdata', this.libds)
.then((res: EditorsGetDataServiceResponse) => {
this.initSetup(res)
})
.catch((err: any) => {
// A synchronous throw inside initSetup (called from the .then()
// above) lands here too, not just a network failure - log it, or
// the table just renders hidden with no clue why.
this.handleGetdataFailure(err)
})
}
}
/**
* Puts the page into its getdata error state. Shared by the callService
* .catch and initSetup's own retry branch: a throw from the retry runs on a
* bare setTimeout, outside that promise chain, so it would otherwise never
* reach the .catch and the user would be left on the loading spinner.
*/
private handleGetdataFailure(err: any) {
// eslint-disable-next-line no-console
console.error('editors/getdata failed:', err)
this.getdataError = true
this.tableTrue = true
}
/**
* VA mode, initial (unfiltered) route only: briefly wait for VA's buffered
* first message and, if it already carries a filter, save that filter and
* navigate to /editor/<table>/<FILTER_RK> so the very first getdata is
* filtered. Returns true if it navigated (caller must abort its own load),
* false to proceed with the normal unfiltered load.
*
* The filter is built directly from VA parameters (param label === DC column,
* which VA↔DC matching already requires; numeric inferred from the VA param
* dataType/value) because DC column metadata isn't available until getdata
* has run. Any failure falls back to the normal load; the post-load replay
* (with full metadata) reconciles to the same filter signature.
*/
private async maybeDeferInitialVaLoad(): Promise<boolean> {
if (!this.isVaEmbed) return false
// Only the initial, unfiltered route — never when a FILTER_RK is present.
if (typeof this.route.snapshot.params['filterId'] !== 'undefined')
return false
const libds = this.route.snapshot.params['libMem']
if (!libds) return false
// VA posts its first message before Angular boots, so it's already in the
// external buffer by now — read it directly (no polling). If it hasn't
// arrived yet, skip the optimisation and let the normal load + live message
// apply the filter.
const msg = this.vaMessaging.latestMessage()
if (!msg) return false
const clauses = this.vaFilter.buildInitialClauses(msg)
if (clauses.length === 0) return false
try {
const res: any = await this.sasStoreService.saveQuery(libds, clauses)
const id = res?.result?.[0]?.FILTER_RK
const table = res?.result?.[0]?.FILTER_TABLE
if (id === undefined || table === undefined) return false
// Seed the cross-reload signature so the post-reload replay is a no-op.
this.vaMessaging.filterSignature = this.vaFilter.signature(clauses)
await this.reloadEditorRoute('/editor/' + table + '/' + id)
return true
} catch {
return false
}
}
ngAfterViewChecked() {
this.scheduleFilterOverflowCheck()
}
/**
* Measures whether the applied-filter clause fits the collapsed single line,
* and shows or hides the chevron accordingly. The measurement is deferred
* because `filterOverflows` and `filterExpanded` are bound in this view -
* setting them inside the change-detection cycle would raise
* ExpressionChangedAfterItHasBeenChecked.
*/
private scheduleFilterOverflowCheck() {
const el = this.infoBar?.nativeElement
if (!el) {
return
}
const text = el.querySelector('.infoBar-text') as HTMLElement | null
if (!text) {
return
}
// re-measure only when the clause, the available width, or the state changes
const key = `${this.queryText}|${text.clientWidth}|${this.filterExpanded}`
if (key === this.filterMeasuredKey) {
return
}
this.filterMeasuredKey = key
setTimeout(() => {
// while expanded the clause wraps, so there is nothing to measure
if (this.filterExpanded) {
return
}
this.filterOverflows = text.scrollWidth > text.clientWidth
})
}
public toggleFilterPanel() {
this.filterExpanded = !this.filterExpanded
}
ngAfterViewInit() {
// Fix ARIA accessibility issues after table initialization
setTimeout(() => {
this.fixAriaAccessibility()
}, 1000)
// Set up event listener for hot table element
// Double click to edit
setTimeout(() => {
if (this.hotTableComponent && this.hotTableComponent.hotInstance) {
const hotElement = this.hotTableComponent.hotInstance.rootElement
if (hotElement) {
hotElement.addEventListener('mousedown', (event: MouseEvent) => {
if (!this.uploadPreview) {
this.hotClicked()
}
setTimeout(() => {
const menuDebugItem: any =
document.querySelector('.debug-switch-item') || undefined
if (menuDebugItem) menuDebugItem.click()
}, 100)
})
}
}
}, 100)
}
/**
* Recomputes `colHeaders` from `headerColumns`/`cols` for the current
* `useLabels` state, and pushes it into a live grid via `updateSettings`
* if one is mounted (no refetch needed).
*/
private applyDisplayColHeaders() {
if (!this.hotInstance || !this.headerColumns.length) return
const colHeaders = getDisplayColHeaders(
this.headerColumns,
this.cols,
this.useLabels
)
this.updateSettingsSortSafe({ colHeaders }, false)
}
ngOnDestroy() {
this._queryParams?.unsubscribe()
// Clean up the MutationObserver
if (this.ariaObserver) {
this.ariaObserver.disconnect()
this.ariaObserver = undefined
}
// Clean up the interval
if (this.ariaCheckInterval) {
clearInterval(this.ariaCheckInterval)
this.ariaCheckInterval = undefined
}
// Stop observing the grid container for resizes
if (this.gridResizeObserver) {
this.gridResizeObserver.disconnect()
this.gridResizeObserver = undefined
}
// Remove the native paste listener (see its own registration comment -
// attached once on rootElement rather than per edit session)
if (this.pasteListener && this.hotInstance?.rootElement) {
this.hotInstance.rootElement.removeEventListener(
'paste',
this.pasteListener
)
this.pasteListener = undefined
}
// Cancel any pending debounced VA apply
if (this.vaDebounceTimer) {
clearTimeout(this.vaDebounceTimer)
this.vaDebounceTimer = undefined
}
// Remove the VA postMessage listener
if (this.vaUnsubscribe) {
this.vaUnsubscribe()
this.vaUnsubscribe = undefined
}
}
/**
* Re-run height+render when the grid container settles.
*/
private observeGridResize() {
const el = this.hotInstance?.rootElement?.parentElement
if (!el) return
this.gridResizeObserver?.disconnect()
this.gridResizeObserver = new ResizeObserver(() => {
requestAnimationFrame(() => {
const hot = this.hotInstance
if (!hot || hot.isDestroyed) return
this.updateSettingsSortSafe({ height: this.hotTable.height }, false)
hot.render()
})
})
this.gridResizeObserver.observe(el)
}
/**
* Fixes ARIA accessibility issues in the Handsontable component
* This addresses the accessibility report issues with treegrid and presentation roles
*/
private fixAriaAccessibility() {
// Use a more aggressive approach to find and fix all ARIA issues
const fixAriaIssues = () => {
// Specifically target Handsontable wrapper elements that are causing issues
const hotWrappers = document.querySelectorAll(
'.ht-wrapper, .wtHolder, [id^="ht_"]'
)
hotWrappers.forEach((wrapper) => {
// Remove problematic ARIA attributes from Handsontable wrappers
wrapper.removeAttribute('role')
wrapper.removeAttribute('aria-rowcount')
wrapper.removeAttribute('aria-colcount')
wrapper.removeAttribute('aria-multiselectable')
})
// Find all elements with problematic ARIA roles in the entire document
const allTreegridElements = document.querySelectorAll('[role="treegrid"]')
const allPresentationElements = document.querySelectorAll(
'[role="presentation"]'
)
// Fix treegrid role issues - remove them completely as they're causing problems
allTreegridElements.forEach((element) => {
element.removeAttribute('role')
element.removeAttribute('aria-rowcount')
element.removeAttribute('aria-colcount')
element.removeAttribute('aria-multiselectable')
})
// Fix presentation role issues - remove them if they contain interactive elements
allPresentationElements.forEach((element) => {
const hasInteractiveChildren =
element.querySelectorAll(
'button, input, select, textarea, [tabindex], [onclick], [contenteditable]'
).length > 0
if (hasInteractiveChildren) {
element.removeAttribute('role')
}
})
// Also fix any elements with aria-rowcount="-1" which is problematic
const negativeRowCountElements = document.querySelectorAll(
'[aria-rowcount="-1"]'
)
negativeRowCountElements.forEach((element) => {
element.removeAttribute('aria-rowcount')
})
// Ensure proper table structure
const tableElements = document.querySelectorAll('table')
tableElements.forEach((table) => {
if (!table.getAttribute('role')) {
table.setAttribute('role', 'table')
}
// Ensure table headers have proper scope
const headerCells = table.querySelectorAll('th')
headerCells.forEach((th) => {
if (!th.getAttribute('scope')) {
th.setAttribute('scope', 'col')
}
})
})
// Add proper ARIA labels to interactive elements
const interactiveElements = document.querySelectorAll(
'button, input, select, textarea, [contenteditable]'
)
interactiveElements.forEach((element) => {
if (
!element.getAttribute('aria-label') &&
!element.getAttribute('aria-labelledby')
) {
const textContent = element.textContent?.trim()
if (textContent) {
element.setAttribute('aria-label', textContent)
}
}
})
}
// Run the fix immediately
fixAriaIssues()
// Run it again after a short delay to catch any dynamically created elements
setTimeout(fixAriaIssues, 100)
setTimeout(fixAriaIssues, 500)
setTimeout(fixAriaIssues, 1000)
setTimeout(fixAriaIssues, 2000)
// Set up a periodic check to ensure accessibility fixes are maintained
if (!this.ariaCheckInterval) {
this.ariaCheckInterval = setInterval(fixAriaIssues, 3000)
}
// Set up a MutationObserver to continuously monitor for new problematic elements
if (!this.ariaObserver) {
this.ariaObserver = new MutationObserver((mutations) => {
let shouldFix = false
mutations.forEach((mutation) => {
if (
mutation.type === 'attributes' &&
(mutation.attributeName === 'role' ||
mutation.attributeName === 'aria-rowcount')
) {
shouldFix = true
}
if (mutation.type === 'childList') {
mutation.addedNodes.forEach((node) => {
if (node.nodeType === Node.ELEMENT_NODE) {
const element = node as Element
if (
element.hasAttribute('role') ||
element.hasAttribute('aria-rowcount')
) {
shouldFix = true
}
}
})
}
})
if (shouldFix) {
setTimeout(fixAriaIssues, 50)
}
})
// Start observing the entire document for changes
this.ariaObserver.observe(document.body, {
childList: true,
subtree: true,
attributes: true,
attributeFilter: [
'role',
'aria-rowcount',
'aria-colcount',
'aria-multiselectable'
]
})
}
}
initSetup(response: EditorsGetDataServiceResponse, attempt = 0) {
if (this.getdataError) return
if (!response || !response.data) return
this.hotInstance = this.hotTableComponent?.hotInstance!
if (!this.hotInstance) {
// Retry init, don't permanently abort. A throw from the retry runs on a
// bare setTimeout rather than in the caller's promise chain, so it would
// never reach that .catch - surface it through the same handler here.
if (attempt < 60) {
setTimeout(() => {
try {
this.initSetup(response, attempt + 1)
} catch (err) {
this.handleGetdataFailure(err)
}
}, 50)
}
return
}
// The delete column is client-owned (see deleteRecordColumnRule.ts):
// this asserts getdata did not send it, then prepends it to COLHEADERS so
// it is the grid's first column. A throw here lands in the caller's
// .catch - or, on a retry, the branch above - which sets getdataError and
// shows the error state.
prepareDeleteRecordColumn(response)
this.cols = response.data.cols
this.dsmeta = response.data.dsmeta
this.versions = response.data.versions || []
const notes = this.dsmeta.find((item) => item.NAME === 'NOTES')
const longDesc = this.dsmeta.find((item) => item.NAME === 'DD_LONGDESC')
const shortDesc = this.dsmeta.find((item) => item.NAME === 'DD_SHORTDESC')
if (notes && notes.VALUE) {
this.dsNote = notes.VALUE
} else if (longDesc && longDesc.VALUE) {
this.dsNote = longDesc.VALUE
} else if (shortDesc && shortDesc.VALUE) {
this.dsNote = shortDesc.VALUE
} else {
this.dsNote = ''
}
// getdata serves a selection over DC_MAXOBS_WEBEDIT / DC_MAXCELLS_WEBEDIT
// truncated to the first rows within the limit, and carries the reason
// here. Empty means the whole selection is in the grid - and a blank
// character arrives as a single blank (mp_jsonout cannot emit a zero-length
// string, since SAS has none), so trim it: whitespace is "no note", not a
// warning with an empty tooltip.
this.truncationNote = (response.data.sasparams[0].TRUNC_MSG || '').trim()
const hot = this.hotInstance
const approvers: Approver[] = response.data.approvers
if (this.cols) {
this.headerArray = parseTableColumns(this.cols)
}
// Note: the above this.headerArray is being reassigned without being used.
// So, above assignment does not make sense.
approvers.forEach((item: Approver) => {
this.approvers.push(item.PERSONNAME)
})
this.tableTrue = true
this.libds = response.libds
this.hotTable.data = response.data.sasdata
this.headerColumns = response.data.sasparams[0].COLHEADERS.split(',')
this.headerPks = response.data.sasparams[0].PK.split(' ')
// Derived (computed) columns, named by getdata from MPE_DERIVED - read-only
// through a READONLY DQ rule, and marked in the grid so they read as
// computed rather than merely non-editable.
this.derivedVars = (response.data.sasparams[0].DERIVEDVARS || '')
.split(' ')
.filter((col: string) => !!col)
this.columnLevelSecurityFlag = !!response.data.sasparams[0].CLS_FLAG
if (this.columnLevelSecurityFlag)
this.setRestrictions({
restrictAddRow: true,
removeEditRecordButton: true,
removeAddRecordButton: true
})
this.checkEmptyRowWhenFilter()
if (this.headerColumns.indexOf(DELETE_RECORD_COLUMN_NAME) !== -1) {
this.headerColumns[
this.headerColumns.indexOf(DELETE_RECORD_COLUMN_NAME)
] = 'Delete?'
}
this.headerArray = this.headerColumns.slice(1)
// EDIT_STATUS is never part of COLHEADERS (it's client-synthesized, see
// editStatusColumnRule.ts) - appended after headerArray is derived so it
// stays out of whatever headerArray drives, but before headerColumns is
// used below for cell-reference math (applyFormulaRules/DcValidator's
// rules must stay index-aligned with headerColumns).
this.headerColumns.push(EDIT_STATUS_COLUMN_NAME)
if (response.data.sasparams[0].DTVARS !== '') {
this.dateHeaders = response.data.sasparams[0].DTVARS.split(' ')
}
if (response.data.sasparams[0].TMVARS !== '') {
this.timeHeaders = response.data.sasparams[0].TMVARS.split(' ')
}
if (response.data.sasparams[0].DTTMVARS !== '') {
this.dateTimeHeaders = response.data.sasparams[0].DTTMVARS.split(' ')
}
if (response.data.xl_rules.length > 0) {
this.xlRules = this.helperService.deepClone(response.data.xl_rules)
}
this.dcValidator = new DcValidator(
response.data.sasparams[0],
response.data.$sasdata,
this.cols,
response.data.dqrules,
response.data.dqdata
)
// this.hotTable.formulas is set once, on the field itself, and applies
// to every table regardless of whether it has any HARDFORMULA/
// SOFTFORMULA rules.
// BASE_COL names of every HARDFORMULA/SOFTFORMULA rule - applyFormulaRules
// below legitimately writes a live `=...` formula string into these, so
// the escape loop further down must skip them (they're never
// backend-sourced text that merely looks like a formula).
const formulaBaseCols = getFormulaColumnNames(response.data.dqrules)
this.cellValidation = this.dcValidator.getRules()
// to take datasource
this.dataSource = response.data.sasdata
this.$dataFormats = response.data.$sasdata
// Raw, as-received snapshot - captured before applyFormulaRules below
// overwrites HARDFORMULA/SOFTFORMULA columns, so markOverwrittenCells
// can later tell "the formula filled in a blank" apart from "the
// formula overwrote a value the real dataset already had".
this.dataSourceRaw = this.helperService.deepClone(this.dataSource)
// Seed every row's EDIT_STATUS to 'U' before anything (HyperFormula
// included) ever sees this data - nothing has been edited yet, and this
// is also the baseline dataSourceUnchanged will later be cloned from, so
// future diffs never see a spurious EDIT_STATUS mismatch (see
// classifyRow's withoutEditStatus for why that would matter).
for (const row of this.dataSource) {
row[EDIT_STATUS_COLUMN_NAME] = 'U'
}
// The delete flag is client-owned too (see deleteRecordColumnRule.ts) -
// getdata no longer sends the column, so every row starts unmarked.
seedDeleteRecordColumn(this.dataSource)
// Seed HARDFORMULA/SOFTFORMULA columns with their computed formula
// string per row. dataSourceUnchanged isn't populated yet this early in
// a fresh load (only the excel-preview/add-row flows set it) - at this
// point, before any edits exist, dataSource IS the unchanged baseline,
// so DC.ORIG_VALUE resolves correctly either way.
applyFormulaRules(
this.dataSource,
response.data.dqrules,
this.headerColumns,
this.dataSourceUnchanged ?? this.dataSource,
this.headerPks,
this.userService.user?.username ?? ''
)
// Prevent a `=`-led value from the backend in any character column from
// being treated as a live formula - see character-column-formula-plan.md.
// Must run after applyFormulaRules above (which legitimately writes a
// live formula string into HARDFORMULA/SOFTFORMULA columns) and before
// dataSourceUnchanged is cloned below, so the "unchanged baseline"
// already reflects the escaped form rather than flagging these cells as
// modified. Only backend-sourced values are ever escaped here - a value
// the user later types or pastes is never touched (see beforeChange).
//
// Also escapes the matching dataSourceRaw cell (still index-aligned
// with dataSource here, before any sort/insert reorders things) for the
// same non-formula columns, so it stays in sync with what the user
// actually sees. dataSourceRaw drives the Revert feature - if it kept
// the true unescaped backend string instead, reverting a cell after
// "Apply as formula" would restore the raw `=...` text, which
// (formulas being enabled everywhere) evaluates live again instead of
// going back to the escaped text the user started from. Formula base
// columns are still skipped, so dataSourceRaw keeps meaning "value
// before the formula overwrote it" for them, same as before.
this.autoEscapedCells = new Map()
this.dataSource.forEach((row, rowIndex) => {
const rowKey = getRowKey(row, this.headerPks)
for (const colName of this.headerColumns) {
if (formulaBaseCols.includes(colName)) continue
if (!isCharacterColumn(this.cols, colName)) continue
const { value, wasEscaped } = escapeCharacterColumnValue(row[colName])
// A row whose backend source never had this column at all (a
// sparse row) must stay that way - writing back unconditionally
// would materialize a new own property set to undefined, which
// the SASjs adapter rejects at submit time even though nothing
// about this cell ever changed.
if (!wasEscaped) continue
row[colName] = value
if (this.dataSourceRaw[rowIndex]) {
this.dataSourceRaw[rowIndex][colName] = value
}
markAutoEscaped(this.autoEscapedCells, rowKey, colName)
}
})
// Seeded here too (not just editTable()) so a HARDFORMULA/SOFTFORMULA
// rule that silently overwrote real pre-existing data (see
// markOverwrittenCells) already shows the row as modified - both the
// '~' row header and EDIT_STATUS='M' - on the very first render, before
// the user ever clicks Edit. editTable() rebuilds this fresh every time
// it runs regardless, so setting it here doesn't affect that.
this.dataSourceUnchanged = this.helperService.deepClone(this.dataSource)
this.overlayFormulaRawValuesOnUnchanged(this.dataSourceUnchanged)
// Note: this.headerColumns and this.columnHeader contains same data
// need to resolve redundancy
// default schema - includes NOTNULL defaults from DQ rules
for (let i = 0; i < this.headerColumns.length; i++) {
const colType = this.cellValidation[i].type
this.hotDataSchema[this.cellValidation[i].data] = getHotDataSchema(
colType,
this.cellValidation[i],
this.dcValidator?.getDqDetails()
)
}
// this.addActionButtons();
// this.addActionColumns();
// now all validation params we have in this.cellValidation
this.checkRowLimit()
hot.updateSettings(
{
data: this.dataSource,
colHeaders: getDisplayColHeaders(
this.headerColumns,
this.cols,
this.useLabels
),
columns: this.cellValidation,
height: this.hotTable.height,
formulas: this.hotTable.formulas,
// The Formulas plugin resolves a =CELL_REF formula (e.g. what
// DC.ROW_STATUS compiles to, see parseFormulaRule.ts) through its own
// data-sync path into HyperFormula - a different path than
// getDataAtRowProp/datamap.get() (see editStatusColumnRule.ts for
// why a dotted `data` key is otherwise safe there). That path
// doesn't check for an own flat property before falling back to
// nested dot-notation, so a referenced cell whose `data` key
// contains a literal '.' (like dc.row_status) resolves to
// undefined/0 instead of its real value. This app never uses nested
// `data` paths, so disabling dot-notation entirely lets
// DC.ROW_STATUS (and anything that branches on it, e.g.
// CHANGE_SUMMARY_COL's IF) resolve correctly without giving up the
// collision-proof dotted name.
dataDotNotation: false,
// readOnly here means comments can never be added/edited/removed
// through the UI - only markOverwrittenCells/
// syncOverwrittenCommentForCell (via the plugin API) ever set one.
// The context menu below only offers our own "Revert" item, never
// the plugin's own add/edit/remove ones.
comments: { readOnly: true },
stretchH: 'all',
readOnly: this.hotTable.readOnly,
hiddenColumns: {
indicators: true,
columns: this.dcValidator.getHiddenColumns()
},
modifyColWidth: function (width: number, col: number) {
if (col === 0) {
return 60
}
if (width > 500) return 500
else return width
},
copyPaste: this.hotTable.copyPaste,
manualColumnFreeze: false, //https://handsontable.com/docs/7.0.3/demo-freezing.html
// false due to https://forum.handsontable.com/t/gh-5112-column-freeze-sorting/3236
multiColumnSorting: true, // https://handsontable.com/docs/7.0.0/demo-multicolumn-sorting.html
manualColumnResize: true,
filters: false,
manualRowResize: true,
viewportRowRenderingOffset: 100,
// Doubles as the edit-status indicator: +/-/~ for
// added/deleted/modified rows. Handsontable re-invokes this on
// every render, so it stays live as rows are edited/added/deleted
// without any extra wiring. Falls back to a plain space (not an
// empty string) for unchanged rows and before dataSource is
// populated, keeping the original row-selection bar's click target
// intact - a space and '' render identically, so this changes
// nothing visually.
//
// `index` is the VISUAL row - once multiColumnSorting reorders the
// grid, that no longer matches dataSource's physical order, so it
// must be translated via toPhysicalRow() before indexing into
// dataSource. Without this, a sorted grid shows every row's symbol
// shifted to the wrong row.
rowHeaders: (index: number) => {
const physicalRow = hot.toPhysicalRow(index)
const dataRow =
physicalRow === null ? undefined : this.dataSource[physicalRow]
if (!dataRow) return ' '
return (
getEditStatusSymbol(
classifyRow(
dataRow,
this.dataSourceUnchanged ?? this.dataSource,
this.headerPks
)
) || ' '
)
},
// Colors the row-header cell to match its edit status (see
// rowHeaders above) - same physicalRow translation for the same
// sorted-grid reason.
afterGetRowHeader: (row: number, th: any) => {
const physicalRow = hot.toPhysicalRow(row)
const dataRow =
physicalRow === null ? undefined : this.dataSource[physicalRow]
if (!dataRow) return
const status = classifyRow(
dataRow,
this.dataSourceUnchanged ?? this.dataSource,
this.headerPks
)
th.classList.remove(
'rowStatusAdded',
'rowStatusDeleted',
'rowStatusModified'
)
if (status === 'A') th.classList.add('rowStatusAdded')
if (status === 'D') th.classList.add('rowStatusDeleted')
if (status === 'M') th.classList.add('rowStatusModified')
},
rowHeaderWidth: 20,
rowHeights: 24,
// Handsontable's own maxRows (a UI/licensing cap on how many NEW
// rows can be ADDED - see checkRowLimit/restrictAddRow) silently
// becomes HyperFormula's OWN sheet-size limit too (see
// getEngineSettingsOverrides in Handsontable's formulas plugin,
// which maps hotSettings.maxRows straight into the engine config).
// editor_rows_allowed can be smaller than a table's ALREADY-
// EXISTING row count, and formulas are on for every table - capping
// at editor_rows_allowed alone would make HyperFormula reject the
// sheet outright for any table bigger than that cap. Never let it
// go below what's actually loaded - the add-row restriction itself
// is still enforced separately, by checkRowLimit's own comparison
// against dataSource.length.
maxRows: Math.max(
this.dataSource.length,
this.licenceState.value.editor_rows_allowed || Infinity
),
invalidCellClassName: 'htInvalid',
// Prevent automatic row creation
autoWrapRow: false,
autoWrapCol: false,
// Ensure proper data binding
bindRowsWithHeaders: false,
dropdownMenu: {
items: {
make_read_only: {
name: 'make_read_only'
},
alignment: {
name: 'alignment'
},
sp1: {
name: '---------'
},
info: {
name: 'test info',
// Purely informational (no callback) - without this, Handsontable
// still treats a click landing anywhere inside it as "the item
// was activated" (see preventMenuItemAutoClose's own comment),
// auto-closing the menu and blocking the native right-click
// "Copy" menu before the user can select any of the text.
isCommand: false,
renderer: (
hot: Handsontable.Core,
wrapper: HTMLElement,
row: number,
col: number,
prop: string | number,
itemValue: string
) => {
const elem = document.createElement('span')
let colName = ''
let colInfo: DataFormat | undefined
let textInfo = 'No info found'
if (this.hotInstance) {
// getSelected() is typed number[][] in HOT 18 (was a 4-tuple
// array before); loosen the annotation to match.
const hotSelected: number[][] =
this.hotInstance.getSelected() || []
const selectedCol: number = hotSelected
? hotSelected[0][1]
: -1
colName = this.hotInstance?.colToProp(selectedCol) as string
colInfo = this.$dataFormats?.vars[colName]
const { hardRegexValue, softRegexValue } =
this.dcValidator?.getRegexRuleValues(colName) || {}
const formulaValue =
this.dcValidator?.getFormulaRuleValue(colName)
textInfo = buildColInfoHtml(
colName,
colInfo,
hardRegexValue,
softRegexValue,
formulaValue
)
}
elem.innerHTML = textInfo
preventMenuItemAutoClose(elem)
return elem
}
}
}
},
// filters: true,
dataSchema: this.hotDataSchema,
contextMenu: this.hotTable.settings.contextMenu,
//, '---------','freeze_column','unfreeze_column'],
currentHeaderClassName: 'customH',
afterGetColHeader: (col: number, th: any) => {
const column = this.columnHeader[col]
// header columns styling - primary keys
const isPKCol = column && this.isColPk(column)
const isReadonlyCol = column && this.isReadonlyCol(column)
const isDerivedCol = column && this.isDerivedCol(column)
if (isPKCol) th.classList.add('primaryKeyHeaderStyle')
if (isReadonlyCol && !isPKCol && !isDerivedCol) {
th.classList.add('readonlyCell')
}
// a derived column is computed, not merely locked - give it its own
// hue and an 'fx' marker rather than the read-only dimming
if (isDerivedCol) {
th.classList.add('dc-derived-header')
// the label says what the column is - for a derived variable that
// is the configured VAR_LABEL, or the generated name=expression
// when the administrator left it blank. A label that merely
// repeats the name adds nothing, so it falls back to the rule.
const derivedLabel = (this.cols || [])
.filter((c: any) => c.NAME === column)
.map((c: any) => String(c.LABEL || '').trim())
.find(
(l: string) =>
l.length > 0 &&
l.toUpperCase() !== String(column).toUpperCase()
)
th.title = derivedLabel
? `${derivedLabel} - derived variable, computed at read time and cannot be edited`
: 'Derived variable - computed at read time and cannot be edited'
const firstDerivedCol = this.columnHeader.findIndex(
(name: string) => this.isDerivedCol(name)
)
if (col === firstDerivedCol) th.classList.add('dc-derived-first')
}
// Remove header arrow from Delete column
if (col === 0) {
th.classList.add('firstColumnHeaderStyle')
}
// Dark mode
th.classList.add(globals.handsontable.darkTableHeaderClass)
},
afterGetCellMeta: (
row: number,
col: number,
cellProperties: Handsontable.CellProperties
) => {
const isReadonlyCol = col && this.isReadonlyCol(col)
const isDerivedCol = col && this.isDerivedCol(col)
// Check if this cell should be marked as invalid due to duplicate primary key values
// Only applies to primary key columns (col 1 through readOnlyFields)
const isDuplicateCell =
this.duplicatePkIndexes.includes(row) &&
col >= 1 &&
col <= this.readOnlyFields
// Handle existing CSS classes - Handsontable can provide className as string or array
const existingClasses = cellProperties.className || ''
let classes: string[]
if (Array.isArray(existingClasses)) {
// If already an array, create a copy
classes = [...existingClasses]
} else {
// If string, split by spaces and filter out empty strings
classes = existingClasses
.split(' ')
.filter((c: string) => c.length > 0)
}
// Add readonlyCell class for readonly columns to maintain original styling
if (isDerivedCol) {
// a derived cell carries its own tint; the read-only dimming would
// wash it out, and "greyed out" is a different message from
// "computed here"
classes = classes.filter((c: string) => c !== 'readonlyCell')
if (!classes.includes('dc-derived-cell')) {
classes.push('dc-derived-cell')
}
// derived columns are appended after the physical ones - mark the
// start of the block so it reads as one group
const firstDerivedCol = this.columnHeader.findIndex(
(name: string) => this.isDerivedCol(name)
)
if (
col === firstDerivedCol &&
!classes.includes('dc-derived-first')
) {
classes.push('dc-derived-first')
}
} else if (isReadonlyCol && !classes.includes('readonlyCell')) {
classes.push('readonlyCell')
}
// Apply custom validation styling for duplicate primary key cells
// Note: Uses 'dc-invalid-cell' instead of Handsontable's 'htInvalid' class
// because Handsontable's internal validation system was removing 'htInvalid'
// causing flickering. Our custom class persists reliably.
if (isDuplicateCell) {
if (!classes.includes('dc-invalid-cell')) {
classes.push('dc-invalid-cell')
}
// Mark cell as invalid to prevent form submission
cellProperties.valid = false
// Custom flag to identify this as a duplicate key cell for cleanup
cellProperties.dupKey = true
}
// Apply the combined CSS classes back to the cell
if (classes.length > 0) {
cellProperties.className = classes.join(' ')
}
}
},
false
)
// Must run after the updateSettings() call above: getDataAtRowProp only
// resolves formulas' live computed values once HyperFormula has
// actually evaluated them against the data/formulas settings just
// applied.
this.markOverwrittenCells()
this.resolvePkFormulaSeedValues()
this.hotTable.hidden = false
// Keep the context menu enabled in view mode too so Copy/Export remain
// available; editing items hide themselves when read-only.
this.toggleHotPlugin('contextMenu', true)
/**
* This is needed if freeze column is enabled
*/
// hot.getPlugin('manualColumnFreeze').freezeColumn(0);
this.queryText = response.data.sasparams[0].FILTER_TEXT
this.columnHeader = response.data.sasparams[0].COLHEADERS.split(',')
// First column is always used to mark records for deletion
this.columnHeader[0] = 'Delete?'
this.readOnlyFields = response.data.sasparams[0].PKCNT
hot.addHook(
'afterSelection',
(
row: number,
column: number,
row2: number,
column2: number,
preventScrolling: any,
selectionLayerLevel: any
) => {
/**
* This is needed if freeze column is enabled
*/
// if (column === 0) {
// delete contextMenuToSet.items.unfreeze_column;
// }
if (
row === row2 &&
column === column2 &&
this.hotTable.readOnly === false
) {
this.dynamicCellValidation(row, column)
}
/**
* This is needed if freeze column is enabled
*/
// if (column === 0) {
// if (!this.firstColumnSelected) {
// hot.updateSettings({
// contextMenu: contextMenuToSet
// });
// this.firstColumnSelected = true;
// }
// } else {
// if (this.firstColumnSelected) {
// hot.updateSettings({
// contextMenu: contextMenuToSet
// });
// this.firstColumnSelected = false;
// }
// }
}
)
hot.addHook('afterBeginEditing', () => {
// When we open a dropdown we want filter disabled so value in cell
// don't filter out items, since we want to see them all.
this.setCellFilter(false)
})
hot.addHook('beforeKeyDown', () => {
// When we start typing, we are enabling the filter since we want to find
// values faster.
this.setCellFilter(true)
})
// ROUND: round numeric values Excel-style before they are written.
// Mutating `changes` in place (rather than setDataAtRowProp) avoids
// re-entrancy and uniformly covers edit, paste and autofill.
hot.addHook('beforeChange', (changes: any[], source: any) => {
if (!changes) return
for (const change of changes) {
if (!change) continue
const [changeRow, prop, , newValue] = change
const colName =
typeof prop === 'string'
? prop
: (hot.colToProp(prop as number) as string)
const digits = this.dcValidator?.getRoundDigits(colName)
if (digits !== undefined) {
const num = Number(newValue)
if (newValue !== null && newValue !== '' && !isNaN(num)) {
change[3] = excelRound(num, digits)
}
}
// A `=`-led value the user types or pastes is never auto-escaped
// (see character-column-formula-plan.md) - it's left exactly as
// entered, which genuinely evaluates as a live formula since
// formulas are enabled for every column. If this cell was
// previously auto-escaped from backend data, the user is now
// supplying their own content, so that marker no longer describes
// what's here - clear it so submit/"Apply as formula" leave this
// cell alone.
const row = this.dataSource[hot.toPhysicalRow(changeRow)]
if (!row) continue
const rowKey = getRowKey(row, this.headerPks)
// Revert (see the revert_cells context-menu item, which tags its
// own write with this source) restores dataSourceRaw's value
// verbatim - for a character column that's the same escaped form
// the initial-load pass would have produced, so it's correct to
// treat it the same way and re-mark it, letting "Apply as
// formula" and the submit-time strip work on it again. A value
// that merely happens to start with `'=` because the user typed
// it themselves never takes this path, since only revert_cells
// uses this source.
if (
source === 'revert' &&
typeof newValue === 'string' &&
newValue.startsWith("'=")
) {
markAutoEscaped(this.autoEscapedCells, rowKey, colName)
} else {
clearAutoEscaped(this.autoEscapedCells, rowKey, colName)
}
}
})
hot.addHook('afterChange', (source: any, change: any) => {
if (change === 'edit') {
const hot = this.hotInstance
const row = source[0][0]
const colProp = source[0][1]
const col = hot.propToCol(colProp) as number
// On edit we enabled filter for this cell, now when editing is finished
// We want filter to be disabled again, to be ready for next dropdown opening.
const cellMeta = hot.getCellMeta(row, col)
if (cellMeta && cellMeta.filter === false)
hot.setCellMeta(row, col, 'filter', true)
// Toggling Delete? changes every other cell's validation exemption
// in this row (see DcValidator.setDefaultValidator) — re-validate so
// invalid highlights clear/reappear immediately instead of only at
// submit time.
if (colProp === '_____DELETE__THIS__RECORD_____') {
hot.validateRows([row], () => hot.render())
}
this.executeDynamicCellValidationIfApplicable(colProp, col, row)
}
})
// Keeps each edited row's EDIT_STATUS cell in sync live, so
// DC.ROW_STATUS-based formulas recalculate immediately - not just on
// insert/delete (already handled at their own call sites) but on any
// direct edit, paste or autofill too. 'loadData' fires on every
// hot.updateSettings({data: ...}) call this component already makes
// (initial load, cancelSubmit, ...) where dataSource is already
// consistent, so it's skipped; 'editStatus' is this same hook's own
// writes (via updateEditStatusForRow), skipped to avoid recursion.
//
// Also keeps each edited cell's "overwritten" comment in sync live -
// markOverwrittenCells only runs once, at initial load, so a direct
// edit made afterward (to a cell that wasn't already overwritten by a
// formula) needs its own comment set here; typing a value back to
// match the original just as readily needs that comment removed again.
hot.addHook('afterChange', (changes: any[], source: any) => {
if (!changes || source === 'loadData' || source === 'editStatus') return
const revertableCols = this.getRevertableColumnNames()
const changedRows = new Set<number>()
for (const change of changes) {
if (!change) continue
// changes gives the VISUAL row (Handsontable's own documented
// afterChange shape) - translate to physical before indexing
// dataSource, same as beforeChange's own hot.toPhysicalRow(changeRow)
// a few lines up. syncOverwrittenCommentForCell/updateEditStatusForRow
// both expect physical (they translate back to visual internally,
// only where a Handsontable API actually needs it).
const [visualRow, prop] = change
if (prop === EDIT_STATUS_COLUMN_NAME) continue
const row = hot.toPhysicalRow(visualRow)
if (row === null) continue
changedRows.add(row)
if (revertableCols.includes(prop)) {
this.syncOverwrittenCommentForCell(row, prop)
}
}
for (const row of changedRows) this.updateEditStatusForRow(row)
})
hot.addHook('afterPaste', async (_data: any, coords: any) => {
// In read-only mode HOT discards the paste itself, so nothing to validate.
if (this.hotTable.readOnly) return
const ranges = (coords as any[]).map((r) => ({
startRow: r.startRow,
startCol: r.startCol,
endRow: r.endRow,
endCol: r.endCol
}))
await this.runBulkValidation(hot, ranges, 'paste')
})
hot.addHook(
'afterAutofill',
async (_fillData: any, _sourceRange: any, targetRange: any) => {
if (this.hotTable.readOnly) return
const { from, to } = targetRange
await this.runBulkValidation(
hot,
[
{
startRow: Math.min(from.row, to.row),
startCol: Math.min(from.col, to.col),
endRow: Math.max(from.row, to.row),
endCol: Math.max(from.col, to.col)
}
],
'autofill'
)
}
)
hot.addHook('afterRender', (isForced: boolean) => {
// Fix ARIA accessibility issues after each render
this.fixAriaAccessibility()
})
// Add a more frequent accessibility fix hook
hot.addHook('afterChange', () => {
// Fix ARIA accessibility issues after any data change
setTimeout(() => {
this.fixAriaAccessibility()
}, 50)
})
hot.addHook('afterCreateRow', (source: any) => {
if (source > this.dataSource.length) {
// don't scroll if row is not added to the end (bottom)
const wtHolder = document.querySelector('.wtHolder')
setTimeout(() => {
if (wtHolder) wtHolder.scrollTop = wtHolder.scrollHeight
})
}
})
// Add hook to prevent unwanted row creation
hot.addHook(
'beforeCreateRow',
(index: number, amount: number, source?: any) => {
// Only allow row creation through the Add Row button or context menu
if (
!this.addingNewRow &&
source !== 'ContextMenu.insert_row_above' &&
source !== 'ContextMenu.insert_row_below'
) {
return false
}
}
)
// Auto-populate NOTNULL default when validation fails due to empty value
hot.addHook(
'afterValidate',
(isValid: boolean, value: any, row: number, prop: string | number) => {
if (isValid || !isEmpty(value)) return
const colName =
typeof prop === 'string'
? prop
: (hot.colToProp(prop as number) as string)
const defaultValue = this.dcValidator?.getNotNullDefaultValue(colName)
if (defaultValue === undefined) return
// Auto-populate using setTimeout to avoid modifying during validation
setTimeout(() => {
if (isEmpty(hot.getDataAtRowProp(row, colName))) {
hot.setDataAtRowProp(row, colName, defaultValue, 'autoPopulate')
}
}, 0)
}
)
// Coerce numeric-column values from string → number so length-check
// and downstream validators see the correct type on the first pass
// (string "3.5" passes `Number(value) === value`, masking float-in-
// short-num errors until the next edit).
const coerceNumericRow = (
row: any[],
startCol: number,
rowMaxLen?: number
): any[] =>
row.map((value: any, index: number) => {
if (rowMaxLen !== undefined && index >= rowMaxLen) return value
const colName = this.columnHeader[startCol + index]
const isColNum = this.$dataFormats?.vars[colName]?.type === 'num'
const specialMissing = isSpecialMissing(value)
if (isColNum && !isNaN(value) && !specialMissing) value = value * 1
return value
})
// A primary key identifies its own row for the rest of the edit session
// (getRowKey, dataModified, classifyRow all match rows by comparing PK
// values) - if it's left holding a live formula, HyperFormula's
// recalculation can shift that identity out from under those matches
// between whenever the formula was entered and whenever saveTable()
// finally resolves it. Unlike any other column, resolve a PK's formula
// to its computed value immediately, right when it's entered (typed,
// pasted, autofilled, or "Apply as formula"'d), so the PK is never
// anything other than a stable, already-computed value going forward.
// Deferred via setTimeout, same as the NOTNULL default-population hook
// above, since writing back through setDataAtRowProp from inside
// afterChange itself would re-enter change processing immediately.
hot.addHook('afterChange', (changes: any[], source: any) => {
if (!changes || source === 'loadData' || source === 'resolvePkFormula')
return
for (const change of changes) {
if (!change) continue
const [row, prop, , newValue] = change
if (typeof newValue !== 'string' || !newValue.startsWith('=')) continue
const colName =
typeof prop === 'string'
? prop
: (hot.colToProp(prop as number) as string)
if (!this.headerPks.includes(colName)) continue
setTimeout(() => {
const computed = hot.getDataAtRowProp(row, colName)
hot.setDataAtRowProp(row, colName, computed, 'resolvePkFormula')
}, 0)
}
})
hot.addHook('beforePaste', (data: any, cords: any) => {
const startCol = cords[0].startCol
const startRow = cords[0].startRow
for (let r = 0; r < data.length; r++) {
data[r] = coerceNumericRow(data[r], startCol)
// Translate column names in a pasted formula (e.g. "=PRICE * VOLUME")
// into this row's cell references ("=B4 * C4") - only meaningful on
// a character column, since that's what's formula-capable for user
// input (see character-column-formula-plan.md). Translating for a
// numeric column would be pointless and would leave a confusing
// half-translated string sitting there instead of the original,
// readable pasted text.
data[r] = data[r].map((value: any, index: number) => {
const colName = this.columnHeader[startCol + index]
if (!isCharacterColumn(this.cols, colName)) return value
return typeof value === 'string' && value.trim().startsWith('=')
? substituteColumnReferences(
value.trim(),
this.headerColumns,
startRow + r
)
: value
})
}
})
// Same column-name -> cell-reference translation as beforePaste above,
// but for pasting directly into an open cell editor (double-click,
// then Cmd+V/Ctrl+V or right-click > Paste) - Handsontable's CopyPaste
// plugin only intercepts a grid-level paste (cell selected but not
// being edited); pasting into the editor's own textarea is a native
// browser paste event the plugin never sees. Listens once on the root
// element (paste events bubble) rather than attaching a new listener
// per edit session, to avoid accumulating listeners across repeated
// edits.
//
// This whole registration only ever runs once per component instance
// in practice (initSetup's own retry-until-ready loop returns before
// reaching this point on every attempt except the one that succeeds,
// and table-switching navigates to a fresh component instance rather
// than reloading in place) - but removing any previous listener before
// replacing the reference costs nothing and keeps that guarantee from
// becoming a silent leak if this assumption ever changes.
if (this.pasteListener) {
hot.rootElement.removeEventListener('paste', this.pasteListener)
}
this.pasteListener = (event: ClipboardEvent) => {
const editor: any = hot.getActiveEditor()
if (!editor || editor.row === null || event.target !== editor.TEXTAREA)
return
// Only on a character column - see beforePaste's own comment above
// for why.
const colName = this.columnHeader[editor.col]
if (!isCharacterColumn(this.cols, colName)) return
const pastedText = event.clipboardData?.getData('text/plain') ?? ''
if (!pastedText.trim().startsWith('=')) return
event.preventDefault()
editor.TEXTAREA.value = substituteColumnReferences(
pastedText.trim(),
this.headerColumns,
editor.row
)
// A real paste's default action would fire this itself once the
// text landed - since preventDefault() above skips that, dispatch it
// manually so the editor's own input-tracking (autosize, its
// internal notion of the current value) picks up the change.
editor.TEXTAREA.dispatchEvent(new Event('input', { bubbles: true }))
}
hot.rootElement.addEventListener('paste', this.pasteListener)
hot.addHook(
'beforeAutofill',
(selectionData: any[][], sourceRange: any) => {
const startCol = sourceRange.from.col
return selectionData.map((row) => coerceNumericRow(row, startCol))
}
)
hot.addHook('afterRemoveRow', () => {
this.checkRowLimit()
})
hot.addHook('afterCreateRow', () => {
this.checkRowLimit()
})
this.uploadUrl = 'services/editors/loadfile'
if (this.recordAction !== null) {
if (this.recordAction === 'ADD') {
this.addRecord()
this.editRecord(null, this.dataSource.length - 1, true)
} else {
if (this.dataSource.length === 1) {
this.editRecord(null, 0)
}
}
}
if (response.data.query.length > 0) {
if (
(globals.rootParam === 'home' || globals.rootParam === 'editor') &&
globals.editor.filter.clauses.length === 0
) {
globals.editor.filter.query = this.helperService.deepClone(
response.data.query
)
globals.editor.filter.libds = this.route.snapshot.params['libMem']
this.sasStoreService.initializeGlobalFilterClause('editor', this.cols)
}
}
hot.render()
this.observeGridResize()
// Fix ARIA accessibility issues after table initialization
setTimeout(() => {
this.fixAriaAccessibility()
}, 500)
// SAS Visual Analytics data-driven content mode: open in read-only view (NOT
// immediate edit — the user clicks Edit to make changes), re-apply any column
// visibility chosen by VA before this (filter) reload, and start receiving VA
// messages over the postMessage interface.
if (this.isVaEmbed) {
if (this.vaMessaging.visibleColumns) {
this.applyVaColumnVisibility(new Set(this.vaMessaging.visibleColumns))
if (this.hotInstance) this.hotInstance.render()
}
// Listener is already registered early (ngOnInit) so VA's initial post is
// not missed; this is a guarded no-op if so. Now that the grid is ready,
// replay the buffered latest message to apply any filter/visibility that
// arrived during load.
this.subscribeToVaMessages()
// Reconcile to the true latest message after a reload — covers a message
// that arrived during the reload gap (caught by va-early, missed by the
// app listener). Signature dedup makes this a no-op if nothing changed.
this.handleVaMessage()
}
}
/**
* Subscribes to the SAS VA data-driven content postMessage channel. VA pushes
* a fresh message on every selection / parameter change. Registered early (in
* ngOnInit) so the initial message isn't missed; messages are buffered on the
* service and applied via a debounce so a burst of control clicks collapses
* into a single reload on the latest message.
*/
private subscribeToVaMessages() {
if (this.vaUnsubscribe) return
this.vaUnsubscribe = this.vaMessaging.onData(() => this.handleVaMessage())
}
/**
* Handles one VA message immediately (NOT debounced): updates column
* visibility and decides the filter. The cheap work (visibility, building the
* filter clauses, deciding pending) runs per message so the "pending"
* indicator can show the instant VA posts; only the expensive APPLY (saveQuery
* + reload) is debounced (live mode) via scheduleVaFilterApply.
*
* Filter data always comes from DC's getdata; VA only drives (a) column
* visibility and (b) the active filter (its params -> DC clauses, applied by
* reloading at /editor/<table>/<FILTER_RK>).
*/
private handleVaMessage() {
const msg = this.vaMessaging.latestMessage()
if (!msg || !this.libds || !this.cellValidation) return
// Grid not ready yet (cellValidation only has the delete column, or none):
// the label->var map would be empty, so every param would fail to map and
// produce an empty filter that wrongly CLEARS the active one — the infinite
// reload loop. Bail; the load-complete replay re-runs once metadata is in.
if (this.cellValidation.length <= 1) return
const labelToVar = this.vaFilter.buildLabelToVarMap(
this.cellValidation,
this.columnHeader
)
// Columns present in the message -> show these, hide the rest. A message
// with no data columns carries no column info, so leave visibility as-is
// rather than hiding everything.
const dataColumns = this.vaMessaging.dataColumns(msg)
let visibilityChanged = false
if (dataColumns.length > 0) {
const matchedVars = new Set<string>()
for (const { column } of dataColumns) {
const key = (column.label ?? column.name ?? '')
.toString()
.trim()
.toLowerCase()
const varName = labelToVar.get(key)
if (varName) matchedVars.add(varName)
}
this.vaMessaging.visibleColumns = Array.from(matchedVars)
visibilityChanged = this.applyVaColumnVisibility(matchedVars)
}
if (visibilityChanged && this.hotInstance) this.hotInstance.render()
this.updateVaFilterText(msg)
const clauses = this.vaFilter.buildClauses(msg, labelToVar, this.cols)
const signature = this.vaFilter.signature(clauses)
if (signature === this.vaMessaging.filterSignature) {
// Selection matches the applied filter — nothing pending. (Don't clear a
// 'loading' status; that belongs to an apply already in flight.)
if (this.vaFilterStatus === 'pending') this.vaFilterStatus = 'idle'
this.vaPendingClauses = null
return
}
// A real change arrived -> pending (shown in both modes, immediately).
this.vaPendingClauses = clauses
this.vaPendingSignature = signature
this.vaFilterStatus = 'pending'
// Filtering repopulates the grid, so it must never run while the user is
// editing (it would wipe their in-progress edits). While in edit mode we
// only surface the pending indicator; the filter is applied on return to
// read-only (see cancelEdit). Live mode auto-advances after the debounce
// settles; confirm waits for the Apply button.
if (this.vaAutoApply && this.hotTable.readOnly) this.scheduleVaFilterApply()
}
/** Live mode: debounce a burst of VA changes into a single apply of the latest. */
private scheduleVaFilterApply() {
if (this.vaDebounceTimer) clearTimeout(this.vaDebounceTimer)
this.vaDebounceTimer = setTimeout(() => {
this.vaDebounceTimer = undefined
this.applyPendingVaFilter()
}, EditorComponent.VA_DEBOUNCE_MS)
}
/**
* Saves the VA-derived filter through DC's validatefilter flow and reloads the
* editor at /editor/<table>/<FILTER_RK> (preserving ?embed=va) so the filter
* key is visible in the URL. Empty clauses clear the filter.
*/
private async applyVaFilter(clauses: QueryClause[], signature: string) {
// A previous filter is still saving/reloading: drop this call WITHOUT
// advancing the signature. The reloaded editor's load-complete replays the
// latest buffered message, which re-evaluates against the still-accurate
// last-applied signature and applies whatever the newest selection is.
if (this.vaApplyingFilter) return
this.vaApplyingFilter = true
// Now actually fetching: surface the "loading filter" status (until DC's
// native "Loading Table" takes over after the reload navigates).
this.vaFilterStatus = 'loading'
// Record the signature only now that we're actually applying it.
this.vaMessaging.filterSignature = signature
try {
if (clauses.length === 0) {
// Already unfiltered (no FILTER_RK in route)? Do nothing — reloading
// would be a pointless round-trip and re-trigger the drain/replay cycle.
// Only reload to clear a filter that is currently applied.
if (typeof this.filter_pk === 'undefined') {
this.vaFilterStatus = 'idle'
return
}
await this.reloadEditorRoute('/editor/' + this.libds)
return
}
const res: any = await this.sasStoreService.saveQuery(
this.libds!,
clauses
)
const id = res?.result?.[0]?.FILTER_RK
const table = res?.result?.[0]?.FILTER_TABLE
if (id === undefined || table === undefined) {
this.vaFilterStatus = 'idle'
return
}
await this.reloadEditorRoute('/editor/' + table + '/' + id)
} catch {
// Keep the current view on failure; the next message can retry.
this.vaFilterStatus = 'idle'
} finally {
this.vaApplyingFilter = false
}
}
/** Apply the staged VA filter (the debounce in live mode, or the Apply button). */
public applyPendingVaFilter(): void {
if (!this.vaPendingClauses) return
// Never apply while editing — the reload would wipe in-progress edits. The
// pending filter stays staged and is applied when the user leaves edit mode.
if (!this.hotTable.readOnly) return
const clauses = this.vaPendingClauses
const signature = this.vaPendingSignature
this.vaPendingClauses = null
void this.applyVaFilter(clauses, signature)
}
/**
* Toggles between live (auto-apply) and confirm filter modes. The choice is
* stored on globals so it survives editor reloads. Switching to live with a
* change still pending applies it immediately.
*/
public toggleVaAutoApply(): void {
globals.vaApplyMode = this.vaAutoApply ? 'confirm' : 'live'
if (this.vaAutoApply && this.vaFilterStatus === 'pending') {
this.applyPendingVaFilter()
}
}
/** Forces a full editor reload of `target`, preserving the embed query param. */
private async reloadEditorRoute(target: string) {
await this.router.navigate(['/'], {
skipLocationChange: true,
queryParamsHandling: 'preserve'
})
await this.router.navigate([target], { queryParamsHandling: 'preserve' })
}
/**
* Shows the DC data columns present in the latest VA message and hides the
* rest (primary-key and validator-hidden columns are always preserved). VA
* parameters can add/remove columns between pushes.
*/
private applyVaColumnVisibility(matchedVars: Set<string>): boolean {
const hot = this.hotInstance
if (!hot) return false
const plugin: any = hot.getPlugin('hiddenColumns')
if (!plugin) return false
const validatorHidden: number[] = this.dcValidator?.getHiddenColumns() || []
const toHide: number[] = []
const toShow: number[] = []
for (let i = 1; i < this.cellValidation.length; i++) {
const varName = this.cellValidation[i]?.data
if (!varName || this.isColPk(varName)) continue
if (validatorHidden.includes(i)) continue
if (matchedVars.has(varName)) {
if (plugin.isHidden(i)) toShow.push(i)
} else if (!plugin.isHidden(i)) {
toHide.push(i)
}
}
if (toShow.length) plugin.showColumns(toShow)
if (toHide.length) plugin.hideColumns(toHide)
return toShow.length > 0 || toHide.length > 0
}
/** Logs the active VA filter summary (kept out of the embedded grid UI). */
private updateVaFilterText(msg: VaMessage) {
const summary = (msg.parameters || [])
.filter((p) => p && p.name !== undefined && p.value !== undefined)
.map((p) => `${p.label ?? p.name}=${p.value}`)
.join(', ')
// eslint-disable-next-line no-console
console.log('VA filters active:', summary)
}
}