Where BatchLAS work runs: a device, a backend, an ordered SYCL queue and a workspace arena.
Queue sibling(ctx,
true);
@ NETLIB
Host CBLAS / LAPACKE; the fallback for any device.
Queue()
A Queue on Device::default_device(), in-order, backend AUTO.
Device device() const
The device this Queue runs on.
Every entry point takes its backend from the Queue and leases its default workspace from the Queue's arena (see WorkspaceLease). Movable, not copyable. Constructing the first Queue in the process closes batchlas::configure().
- Warning
- A Queue is single-threaded: it owns an unsynchronised arena and a cached last event, and workspace(), trim_workspace(), submissions, enqueue(), get_event() and create_event_after_external_work() throw batchlas::api_misuse when called from a thread other than the owner. Use one Queue per thread, or hand one over with attach_to_current_thread().
- See also
- "Synchronisation and threading" in docs/cpp-api.md
Definition at line 321 of file sycl-device-queue.hh.
#include <batchlas/util/sycl-device-queue.hh>
◆ Queue() [1/6]
| batchlas::Queue::Queue |
( |
| ) |
|
◆ ~Queue()
| batchlas::Queue::~Queue |
( |
| ) |
|
◆ Queue() [2/6]
| batchlas::Queue::Queue |
( |
Device | device, |
|
|
bool | in_order = true ) |
A Queue on device with backend AUTO.
- Parameters
-
| device | the device |
| in_order | true (default) for an in-order queue; out-of-order queues make arena-backed workspace releases drain the queue |
◆ Queue() [3/6]
◆ Queue() [4/6]
| batchlas::Queue::Queue |
( |
const Queue & | base, |
|
|
bool | in_order ) |
A Queue sharing base's SYCL context and device, so USM pointers and workspaces allocated for base stay usable.
It has its own arena and event chain.
◆ Queue() [5/6]
| batchlas::Queue::Queue |
( |
Queue && | other | ) |
|
◆ Queue() [6/6]
| batchlas::Queue::Queue |
( |
const Queue & | other | ) |
|
|
delete |
◆ operator=() [1/2]
| Queue & batchlas::Queue::operator= |
( |
Queue && | other | ) |
|
◆ operator=() [2/2]
| Queue & batchlas::Queue::operator= |
( |
const Queue & | other | ) |
|
|
delete |
◆ operator->()
| QueueImpl * batchlas::Queue::operator-> |
( |
| ) |
const |
◆ operator*()
| QueueImpl & batchlas::Queue::operator* |
( |
| ) |
const |
◆ enqueue() [1/2]
| void batchlas::Queue::enqueue |
( |
Event & | event | ) |
|
Order all later work on this Queue after event, without a host wait.
A moved-from Event is ignored; a default-constructed one is already complete.
◆ get_event()
| Event batchlas::Queue::get_event |
( |
| ) |
const |
An Event that completes after everything enqueued on this Queue so far.
◆ create_event_after_external_work()
| Event batchlas::Queue::create_event_after_external_work |
( |
| ) |
|
Like get_event(), but also ordered after work submitted directly to the native stream (cuBLAS, rocBLAS, your own kernels) that bypassed SYCL submission.
◆ enqueue() [2/2]
template<typename EventContainer>
| void batchlas::Queue::enqueue |
( |
EventContainer & | events | ) |
|
|
inline |
◆ wait()
| void batchlas::Queue::wait |
( |
| ) |
const |
Block until all work enqueued on this Queue has completed.
◆ wait_and_throw()
| void batchlas::Queue::wait_and_throw |
( |
| ) |
const |
As wait(), then rethrow any asynchronous error the device reported.
◆ attach_to_current_thread()
| void batchlas::Queue::attach_to_current_thread |
( |
| ) |
|
Transfer single-thread ownership to the calling thread (a move, not a share).
- Precondition
- No other thread uses the Queue during or after the call (unchecked).
- Exceptions
-
◆ workspace()
Borrow bytes of device scratch from this Queue's arena.
The Queue owns the memory: it stays valid until the lease is released, not until the caller returns. Keep the lease in a named variable (auto ws = ctx.workspace(n);).
◆ workspace_capacity()
| size_t batchlas::Queue::workspace_capacity |
( |
| ) |
const |
Bytes the arena currently holds. It grows to the peak requested and keeps it.
◆ trim_workspace()
| bool batchlas::Queue::trim_workspace |
( |
| ) |
|
|
nodiscard |
Return the arena's memory to the runtime (otherwise it is freed only in ~Queue).
- Returns
- false, doing nothing, while any lease is outstanding
- Note
- Waits for the queue to be idle first, so it can throw an asynchronous error.
◆ device()
| Device batchlas::Queue::device |
( |
| ) |
const |
|
inline |
◆ in_order()
| bool batchlas::Queue::in_order |
( |
| ) |
const |
|
inline |
◆ backend()
The backend calls dispatch to; never Backend::AUTO (an AUTO Queue resolves once, on first query, from the device vendor and the compiled-in backends).
- Exceptions
-
◆ requested_backend()
◆ set_backend()
◆ backend_available()
True when backend is compiled into this build.
◆ is_device_accessible()
| bool batchlas::Queue::is_device_accessible |
( |
const void * | ptr | ) |
const |
True for any USM allocation in this Queue's context, and for any non-null pointer on a host/CPU device.
◆ require_device_accessible()
| void batchlas::Queue::require_device_accessible |
( |
const void * | ptr, |
|
|
const char * | what ) const |
As is_device_accessible(), but throws.
- Parameters
-
| ptr | the pointer to check |
| what | names the call-site parameter in the message, e.g. "gemm: A" |
- Exceptions
-
◆ native_handle()
| void * batchlas::Queue::native_handle |
( |
| ) |
const |
The native stream as an opaque pointer: a CUstream when the device runs on the CUDA SYCL backend, a hipStream_t on HIP, nullptr elsewhere.
Keyed off the device, not backend(). Owned by the Queue: do not destroy it or use it past the Queue's lifetime. Work pushed to it runs after BatchLAS's; to make BatchLAS wait for that work, call create_event_after_external_work().
◆ impl_
| std::unique_ptr<QueueImpl> batchlas::Queue::impl_ |
1.18.0
