scalesim.memory package

Submodules

scalesim.memory.double_buffered_scratchpad_mem module

class scalesim.memory.double_buffered_scratchpad_mem.double_buffered_scratchpad

Bases: object

Class which runs the memory simulation of double buffered scratchpad memories (SRAMs). The double buffering helps to hide the DRAM latency when the SRAM is servicing requests from the systolic array using one of the buffers while the other buffer prefetches from the DRAM.

get_dram_trace_matrices()

Method to get the ifmap, filter and ofmap DRAM trace matrices

Returns:

Ifmap, filter and ofmap DRAM trace matrices

get_filter_dram_details()

Method to get the start cycle, stop cycle and number of reads of DRAM requests made by the filter SRAM if trace_valid flag is set.

Returns:

Start cycle, stop cycle and number of reads of DRAM requests made by the filter SRAM

get_filter_dram_trace_matrix()

Method to get the filter DRAM trace matrix. It contains addresses requsted by the filter SRAM and the cycles (first column) at which the requests are made.

Returns:

Filter DRAM trace matrix

get_filter_sram_start_stop_cycles()

Method to get the start and stop cycles of filter SRAM requests by the systolic array if trace_valid flag is set.

Returns:

Start and stop cycles of filter SRAM requests

get_filter_sram_trace_matrix()

Method to get the filter SRAM trace matrix. It contains addresses requsted by the systolic array and the cycles (first column) at which the requests are made.

Returns:

Filter SRAM trace matrix

get_ifmap_dram_details()

Method to get the start cycle, stop cycle and number of reads of DRAM requests made by the ifmap SRAM if trace_valid flag is set.

Returns:

Start cycle, stop cycle and number of reads of DRAM requests made by the ifmap SRAM

get_ifmap_dram_trace_matrix()

Method to get the ifmap DRAM trace matrix. It contains addresses requsted by the ifmap SRAM and the cycles (first column) at which the requests are made.

Returns:

Ifmap DRAM trace matrix

get_ifmap_sram_start_stop_cycles()

Method to get the start and stop cycles of ifmap SRAM requests by the systolic array if trace_valid flag is set.

Returns:

Start and stop cycles of ifmap SRAM requests

get_ifmap_sram_trace_matrix()

Method to get the ifmap SRAM trace matrix. It contains addresses requsted by the systolic array and the cycles (first column) at which the requests are made.

Returns:

Ifmap SRAM trace matrix

get_ofmap_dram_details()

Method to get the start cycle, stop cycle and number of writes of DRAM requests made by the ofmap SRAM if trace_valid flag is set.

Returns:

Start cycle, stop cycle and number of writes of DRAM requests made by the ofmap SRAM

get_ofmap_dram_trace_matrix()

Method to get the ofmap DRAM trace matrix. It contains addresses requsted by the ofmap SRAM and the cycles (first column) at which the requests are made.

Returns:

Ofmap DRAM trace matrix

get_ofmap_sram_start_stop_cycles()

Method to get the start and stop cycles of ofmap SRAM requests by the systolic array if trace_valid flag is set.

Returns:

Start and stop cycles of ofmap SRAM requests

get_ofmap_sram_trace_matrix()

Method to get the ofmap SRAM trace matrix. It contains addresses requsted by the systolic array and the cycles (first column) at which the requests are made.

Returns:

Ofmap SRAM trace matrix

get_sram_trace_matrices()

Method to get the ifmap, filter and ofmap SRAM trace matrices

Returns:

Ifmap, filter and ofmap SRAM trace matrices

get_stall_cycles()

Method to get the number of stall cycles if trace_valid flag is set.

Returns:

Number of stall cycles

get_total_compute_cycles()

Method to get the total number of compute cycles if trace_valid flag is set.

Returns:

Total number of compute cycles

print_filter_dram_trace(filename)

Method to write the filter DRAM trace matrix to a file.

Parameters:

filename – Name of the trace file

Returns:

None

print_filter_sram_trace(filename)

Method to write the filter SRAM trace matrix to a file if trace_valid flag is set.

Parameters:

filename – Name of the trace file

Returns:

None

print_ifmap_dram_trace(filename)

Method to write the ifmap DRAM trace matrix to a file.

Parameters:

filename – Name of the trace file

Returns:

None

print_ifmap_sram_trace(filename)

Method to write the ifmap SRAM trace matrix to a file if trace_valid flag is set.

Parameters:

filename – Name of the trace file

Returns:

None

print_ofmap_dram_trace(filename)

Method to write the iomap DRAM trace matrix to a file.

Parameters:

filename – Name of the trace file

Returns:

None

print_ofmap_sram_trace(filename)

Method to write the Ofmap SRAM trace matrix to a file if trace_valid flag is set.

Parameters:

filename – Name of the trace file

Returns:

None

reset_buffer_states()

Method to reset ifmap, filter and ofmap SRAMs.

Returns:

None

service_filter_reads(incoming_requests_arr_np, incoming_cycles_arr)

Method to service filter read requests coming from systolic array.

Parameters:

• incoming_requests_arr_np – matrix containg address of the memory requsts made from systolic array

• incoming_cycles_arr – list containg cycles at which the memory requsts are made from systolic array

Returns:

A list of out cycles when the requests are serviced

service_ifmap_reads(incoming_requests_arr_np, incoming_cycles_arr)

Method to service ifmap read requests coming from systolic array.

Parameters:

• incoming_requests_arr_np – matrix containg address of the memory requsts made from systolic array

• incoming_cycles_arr – list containg cycles at which the memory requsts are made from systolic array

Returns:

A list of out cycles when the requests are serviced

service_memory_requests(ifmap_demand_mat, filter_demand_mat, ofmap_demand_mat)

Method to run the memory simulation of ifmap, filter and ofmap SRAMs together and generate the traces.

Parameters:

• ifmap_demand_mat – Ifmap demand matrix generated during the compute simulation

• filter_demand_mat – Filter demand matrix generated during the compute simulation

• ofmap_demand_mat – Ofmap demand matrix generated during the compute simulation

Returns:

None

service_memory_requests_old(ifmap_demand_mat, filter_demand_mat, ofmap_demand_mat)

service_ofmap_writes(incoming_requests_arr_np, incoming_cycles_arr)

Method to service ofmap write requests coming from systolic array.

Parameters:

• incoming_requests_arr_np – matrix containg address of the memory requsts made from systolic array

• incoming_cycles_arr – list containg cycles at which the memory requsts are made from systolic array

Returns:

A list of out cycles when the requests are serviced

set_params(verbose=True, estimate_bandwidth_mode=False, word_size=1, ifmap_buf_size_bytes=2, filter_buf_size_bytes=2, ofmap_buf_size_bytes=2, rd_buf_active_frac=0.5, wr_buf_active_frac=0.5, ifmap_backing_buf_bw=1, filter_backing_buf_bw=1, ofmap_backing_buf_bw=1)

Method to set the double buffered memory simulation parameters for housekeeping.

Parameters:

• verbose – Flag to indicate verbosity of memory simulation. If set to False, no output is generated on console

• estimate_bandwidth_mode – Flag to indicate if estimate bandwidth mode is set. If set to false, use user bandwidth mode is set

• word_size – The word size of individual elements

• ifmap_buffer_size_bytes – Ifmap buffer (SRAM) size in bytes

• filter_buffer_size_bytes – Filter (SRAM) size in bytes

• ofmap_buffer_size_bytes – Ofmap (SRAM) size in bytes

• rd_buf_active_fraction – The active fraction of the double duffered ifmap/filter memory (serving the systolic array memory requests)

• rd_buf_active_fraction – The active fraction of the double duffered ofmap memory (serving the systolic array memory requests)

• ifmap_backing_buf_bw – Bandwidth of the backing buffer for ifmap SRAM. The default backing buffer is a dummy one.

• filter_backing_buf_bw – Bandwidth of the backing buffer for filter SRAM. The default backing buffer is a dummy one.

• ofmap_backing_buf_bw – Bandwidth of the backing buffer for ofmap SRAM. The default backing buffer is a dummy one.

Returns:

None

set_read_buf_prefetch_matrices(ifmap_prefetch_mat=array([[0.]]), filter_prefetch_mat=array([[0.]]))

Method to read ifmap and filter prefetch matrices generated in the compute simulation.

Returns:

None

scalesim.memory.read_buffer module

class scalesim.memory.read_buffer.read_buffer

Bases: object

Class which runs the memory simulation of double buffered ifmap/filter SRAM. The double buffering helps to hide the DRAM latency when the SRAM is servicing requests from the systolic array using one of the buffers while the other buffer prefetches from the DRAM.

active_buffer_hit(addr)

Method to check if the address is hit or miss in the active read buffer.

Parameters:

addr – Address of the incoming memory request

Returns:

True if address is hit and false if miss

get_external_access_start_stop_cycles()

Method to get start and stop cycles of the read buffer if trace_valid flag is set.

Returns:

Start and stop cycles of the read buffer

get_hit_latency()

Method to get hit latency of the read buffer.

Returns:

Hit latency of the read buffer

get_latency()

Method to get hit latency of the read buffer.

Returns:

Hit latency of the read buffer

get_num_accesses()

Method to get number of accesses of the read buffer if trace_valid flag is set.

Returns:

Number of accesses of the read buffer

get_trace_matrix()

Method to get the read buffer trace matrix. It contains addresses requsted by the systolic array and the cycles (first column) at which the requests are made.

Returns:

Read buffer trace matrix

new_prefetch()

Method to do a new prefetch. In a new prefetch, some portion of the original data needs to be deleted to accomodate the prefetched data In this case we overwrite some data in the active buffer with the prefetched data and then create a new prefetch request

Returns:

None

prefetch_active_buffer(start_cycle)

Method to prefetch the active read buffer before servicing individual memory requests

Returns:

None

prepare_hashed_buffer()

Method to convert the fetch matrix into a hashed buffer for fast lookups.

Returns:

None

print_trace(filename)

Method to write the read buffer trace matrix to a file.

Parameters:

filename – Name of the trace file

Returns:

None

reset()

Method to reset the read buffer parameters.

Returns:

None

service_reads(incoming_requests_arr_np, incoming_cycles_arr)

Method to service read requests coming from systolic array. Logic: Always check if an addr is in active buffer. If hit, return with hit latency Else, make the contents of prefetch buffer as active and then check Continue making new prefetches until there is a hit

Parameters:

• incoming_requests_arr_np – matrix containg address of the memory requsts made from systolic array

• incoming_cycles_arr – list containg cycles at which the memory requsts are made from systolic array

Returns:

A list of out cycles when the requests are serviced

set_fetch_matrix(fetch_matrix_np)

Method to set the fetch matrix responsible for prefetching from the DRAM

Returns:

None

set_params(backing_buf_obj, total_size_bytes=1, word_size=1, active_buf_frac=0.9, hit_latency=1, backing_buf_bw=1)

Method to set the ifmap/filter double buffered memory simulation parameters for housekeeping.

Parameters:

• backing_buf_obj – Backing buffer object, by default is read_port

• total_size_bytes – Read buffer (SRAM) total size in bytes

• word_size – The word size of individual elements

• active_buf_frac – The active fraction of the double duffered ifmap/filter memory (serving the systolic array memory requests)

• hit_latency – Hit latency of the double duffered ifmap/filter memory

• backing_buf_bw – Bandwidth of the backing buffer for ifmap SRAM. The default backing buffer is a dummy one (read port).

Returns:

None

scalesim.memory.read_buffer_estimate_bw module

class scalesim.memory.read_buffer_estimate_bw.ReadBufferEstimateBw

Bases: object

Class which service the estimate bandwidth mode in the read buffer

check_hit(addr)

Method to check if the address is hit or miss in the active read buffer

Parameters:

addr – Address of the incoming memory request

Returns:

True if address is hit and false if miss

complete_all_prefetches()

Method to complete all the prefetches in estimate bandwidth mode. Prefetch first the active buffer if not done before and then keep prefetching prefetch buffers.

Returns:

None

get_external_access_start_stop_cycles()

Method to get start and stop cycles of the read estimate buffer if trace_valid flag is set.

Returns:

Start and stop cycles of the read estimate buffer

get_hit_latency()

Method to get hit latency of the read estimate buffer.

Returns:

Hit latency of the read estimate buffer

get_latency()

Method to get hit latency of the read estimate buffer.

Returns:

Hit latency of the read estimate buffer

get_num_accesses()

Method to get number of accesses of the read estimate buffer if trace_valid flag is set.

Returns:

Number of accesses of the read estimate buffer

get_trace_matrix()

Method to get the read estimate buffer trace matrix. It contains addresses requsted by the systolic array and the cycles (first column) at which the requests are made.

Returns:

Read estimate buffer trace matrix

manage_prefetches(cycle, addr)

Method to manage prefetches in estimate bandwidth mode

Parameters:

• addr – address of the memory requst made from systolic array

• cycle – cycle at which the memory requst is made from systolic array

Returns:

None

prefetch()

Method to do a new prefetch. In a new prefetch, some portion of the original data needs to be deleted to accomodate the prefetched data In this case we overwrite some data in the active buffer with the prefetched data and then create a new prefetch request

Returns:

None

print_trace(filename)

Method to write the read estimate buffer trace matrix to a file.

Parameters:

filename – Name of the trace file

Returns:

None

service_reads(incoming_requests_arr_np, incoming_cycles_arr)

Method to service read requests coming from systolic array in estimate bandwidth mode

Parameters:

• incoming_requests_arr_np – matrix containg address of the memory requsts made from systolic array

• incoming_cycles_arr – list containg cycles at which the memory requsts are made from systolic array

Returns:

A list of out cycles when the requests are serviced

set_params(backing_buf_obj, total_size_bytes=1, word_size=1, active_buf_frac=0.9, hit_latency=1, backing_buf_default_bw=1)

Method to set the ifmap/filter double buffered memory simulation parameters for estimate bandwidth mode.

Parameters:

• backing_buf_obj – Backing buffer object, by default is read_port

• total_size_bytes – Read buffer (SRAM) total size in bytes

• word_size – The word size of individual elements

• active_buf_frac – The active fraction of the double duffered ifmap/filter memory (serving the systolic array memory requests)

• hit_latency – Hit latency of the double duffered ifmap/filter memory

• backing_buf_bw – Bandwidth of the backing buffer for ifmap SRAM. The default backing buffer is a dummy one (read port).

Returns:

None

scalesim.memory.read_port module

class scalesim.memory.read_port.read_port

Bases: object

Class to define dummy memory like interface to service the requests of the last level memory

get_latency()

Method to get the backing buffer hit latency for housekeeping.

Returns:

Hit latency of the backing buffer

service_reads(incoming_requests_arr_np, incoming_cycles_arr)

Method to service read request by the read buffer. As a dummy memory, everything is a hit

Parameters:

• incoming_requests_arr_np – matrix containg address of the memory requsts made from systolic array

• incoming_cycles_arr – list containg cycles at which the memory requsts are made from systolic array

Returns:

List of out cycles by adding the hit latency to the incoming cycles

set_params(latency)

Method to set the backing buffer hit latency for housekeeping.

Parameters:

latency – Hit latency of the backing buffer

Returns:

None

scalesim.memory.write_buffer module

class scalesim.memory.write_buffer.write_buffer

Bases: object

Class which runs the memory simulation of the ofmap SRAM.

append_to_trace_mat(force=False)

Method to append to the trace matrix cache

Parameters:

force – This flag forces the contents for self.current_line and self.trace_matrix cache to be dumped

Returns:

None

empty_all_buffers(cycle)

Method to drain all of the active buffer.

Parameters:

incoming_cycles_arr – Cycle at which the drain starts

Returns:

None

empty_drain_buf(empty_start_cycle=0)

Method to drain the drain buffer once the active buffer is full.

Parameters:

incoming_cycles_arr – Cycle at which the drain starts

Returns:

Cycle when the drain is completed

get_external_access_start_stop_cycles()

Method to get start and stop cycles of the write buffer if trace_valid flag is set.

Returns:

Start and stop cycles of the write buffer

get_free_space()

Method to get free space of the write buffer.

Returns:

Free space of the write buffer

get_num_accesses()

Method to get number of accesses of the write buffer if trace_valid flag is set.

Returns:

Number of accesses of the write buffer

get_trace_matrix()

Method to get the write buffer trace matrix. It contains addresses requsted by the systolic array and the cycles (first column) at which the requests are made.

Returns:

Write buffer trace matrix

print_trace(filename)

Method to write the write buffer trace matrix to a file.

Parameters:

filename – Name of the trace file

Returns:

None

reset()

Method to reset the write buffer parameters.

Returns:

None

service_writes(incoming_requests_arr_np, incoming_cycles_arr_np)

Method to service write requests coming from systolic array. Logic: Assuming no miss, keep adding to the active buffer. Once the active buffer is full, drain a part of it to the DRAM

Parameters:

• incoming_requests_arr_np – matrix containg address of the memory requsts made from systolic array

• incoming_cycles_arr – list containg cycles at which the memory requsts are made from systolic array

Returns:

A list of out cycles when the requests are serviced

set_params(backing_buf_obj, total_size_bytes=128, word_size=1, active_buf_frac=0.9, backing_buf_bw=100)

Method to set the ofmap memory simulation parameters for housekeeping.

Parameters:

• backing_buf_obj – Backing buffer object, by default is write_port

• total_size_bytes – Write buffer (SRAM) total size in bytes

• word_size – The word size of individual elements

• active_buf_frac – The active fraction of the double duffered ofmap (serving the systolic array memory requests)

• hit_latency – Hit latency of the ofmap memory

• backing_buf_bw – Bandwidth of the backing buffer for ofmap SRAM. The default backing buffer is a dummy one (write port).

Returns:

None

store_to_trace_mat_cache(elem)

Method to add the incoming element to the trace matrix cache

Parameters:

elem – Element requested by the systolic array

Returns:

None

scalesim.memory.write_port module

class scalesim.memory.write_port.write_port

Bases: object

Class to define dummy memory like interface to service the requests of the last level memory

service_writes(incoming_requests_arr_np, incoming_cycles_arr_np)

Method to service read request by the dummy write buffer.

Parameters:

• incoming_requests_arr_np – matrix containg address of the memory requsts made from systolic array

• incoming_cycles_arr_np – list containg cycles at which the memory requsts are made from systolic array

Returns:

List of out cycles

Module contents