Commit Graph

220 Commits

Author SHA1 Message Date
Jason A. Donenfeld
18cfd522aa Use explicit running boolean and use set instead of exchange
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-31 13:59:54 +00:00
Simon Rozman
999a6744db Assert that adapter is always running when client is registering rings
By attaching to NDIS device instead of creating our own device for I/O,
the adapter is always running before client is able to connect and
register rings. NDIS also won't allow adapter to pause with connected
clients.

Signed-off-by: Simon Rozman <simon@rozman.si>
2019-07-31 13:59:54 +00:00
Simon Rozman
7f49a66494 Cleanup TUN_FLAGS_PRESENT
With no PnP notifications and custom surprise removal code we do not
need the TUN_FLAGS_PRESENT any more. The traffic is stopped when handle
is closed or adapter is somehow paused. Though by reusing the NDIS device
for our I/O, the adapter will not be able to pause with a client
connected.

Signed-off-by: Simon Rozman <simon@rozman.si>
2019-07-31 13:59:54 +00:00
Jason A. Donenfeld
5239d53db0 Add handle closing ioctl
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-31 13:59:54 +00:00
Simon Rozman
0d9b9b925a Reinitialize active NBL remove lock to allow adapter reuse
Before the TunProcessReceiveData() thread terminates or adapter is
paused, we wait for all receive NBLs to be returned. Unfortunately,
IoReleaseRemoveLockAndWait() leaves the remove lock in non reusable
state.

To be able to start receiving packets on existing adapter again, we
(re)initialize the remove lock on ring registration or adapter resume.
The former addresses TunProcessReceiveData()'s
IoReleaseRemoveLockAndWait() call, the later addresses the TunPause()'s.

Signed-off-by: Simon Rozman <simon@rozman.si>
2019-07-31 13:59:46 +00:00
Jason A. Donenfeld
f48b0b2883 Compile on 32-bit and arm64
No popcnt intrinsic on arm, no PopulationCount64 function on 32bit.

Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-19 10:28:02 +00:00
Jason A. Donenfeld
970e22d8e4 Use more specific IOCTL code
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-19 06:31:26 +00:00
Jason A. Donenfeld
6ebdbf77d2 Require the usual SDDL_DEVOBJ_SYS_ALL permissions
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-18 19:35:41 +00:00
Simon Rozman
14e5532dc8 Make receiving NBLs asynchronous
This commit moves NBL post-processing (moving ring head, releasing NBL)
to MINIPORT_RETURN_NET_BUFFER_LISTS handler.

Signed-off-by: Simon Rozman <simon@rozman.si>
2019-07-18 15:27:18 +02:00
Simon Rozman
f5eadb50c4 Minimize TransitionLock when receiving packets
We do not need to share-lock the TransitionLock for the whole life of
receiver thread.

Signed-off-by: Simon Rozman <simon@rozman.si>
2019-07-18 14:53:38 +02:00
Jason A. Donenfeld
9bf25d873b Fix insane coding style
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-18 12:49:32 +00:00
Jason A. Donenfeld
cdf0d0bf9a Fix awkward comment style
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-18 12:45:34 +00:00
Jason A. Donenfeld
5502b77218 Do not hijack PNP notifier
We no longer pend send-side NBLs, so we don't have a real use for this.

Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-18 12:41:04 +00:00
Jason A. Donenfeld
aac642e3dc Switch to paged dispatch handlers
We're not totally sure this is kosher, unfortunately.

Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-18 11:28:25 +00:00
Simon Rozman
043abc0a8e Minimize TransitionLock when sending packets
We do not need to share-lock the TransitionLock for complete NBL chain.
This commit should improve better state transition response, thou until
NDIS is sending a single NBL per MINIPORT_SEND_NET_BUFFER_LISTS call,
this should not have a considerable effect.

Since the skibNbl: call of NdisMSendNetBufferListsComplete() is made
inside the TransactionLock at dispatch IRQL, a dispatch IRQL hint was
added to the NdisMSendNetBufferListsComplete() call.

Signed-off-by: Simon Rozman <simon@rozman.si>
2019-07-18 13:52:16 +02:00
Jason A. Donenfeld
58ce3c5000 Piggy-back on top of NDIS' device object instead of adding our own
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-18 10:34:38 +00:00
Jason A. Donenfeld
bf51c91e3e README: clarify ring usage
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-17 21:13:39 +00:00
Simon Rozman
82ede2f96d Replace TUN_FLAGS_CONNECTED by an event
On client closing the handle there is a potential race if somebody
resets the TailMoved event of receive ring at the right time. Rather
than rely on TailMoved event, we introduce our own Disconnected
notification event and have the receive thread wait on both. The
Disconnected  event is also usable as TUN_FLAGS_CONNECTED substitute.

Signed-off-by: Simon Rozman <simon@rozman.si>
2019-07-17 14:53:09 +02:00
Simon Rozman
e7fad38a07 Improve lock retention when sending
NDIS may call MINIPORT_SEND_NET_BUFFER_LISTS from parallel threads to
queue as many packets as fast as possible. Initial implementation of
ring buffers used a spin lock to completely serialize sending packets
making it sub-optimal and burning large amount of CPU.

This commit uses locked section to allocate space for packet(s) in the
ring. It copies the packets unlocked, then it locks again to adjust the
ring tail.

Signed-off-by: Simon Rozman <simon@rozman.si>
2019-07-17 12:22:48 +02:00
Jason A. Donenfeld
1914547ab3 Spin for a bit before falling back to event object
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-17 10:17:46 +00:00
Simon Rozman
66e51bd08f Switch to ring buffers for exchanging packets
This demonstrates the use of ring buffers in its simplest, purest form.
No performance optimizations were made.

Signed-off-by: Simon Rozman <simon@rozman.si>
2019-07-16 20:06:54 +00:00
Jason A. Donenfeld
830df36cd3 README: correct path of device
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Reported-by: Shay Sharon <Shay.Sharon@audiocodes.com>
2019-07-09 11:10:17 +02:00
Simon Rozman
39939e3550 Rename remaining status -> Status and ctx -> Ctx
Signed-off-by: Simon Rozman <simon@rozman.si>
2019-07-08 11:10:43 +02:00
Jason A. Donenfeld
fb738767a0 Remove TunMapIrp
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-05 16:29:00 +00:00
Jason A. Donenfeld
17a0e2647a Simplify ActiveNblCount decrement
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-05 16:29:00 +00:00
Odd Stranne
50aa1f924c Rename NBL -> Nbl
Signed-off-by: Odd Stranne <odd@mullvad.net>
2019-07-05 16:28:59 +00:00
Odd Stranne
d268f03bc1 Rename labels to be more consistent
Signed-off-by: Odd Stranne <odd@mullvad.net>
2019-07-05 16:28:59 +00:00
Odd Stranne
75e434e3b8 Make error branching more compact
Signed-off-by: Odd Stranne <odd@mullvad.net>
2019-07-05 16:28:59 +00:00
Jason A. Donenfeld
20586e37b2 Treat ReferenceCount as an atomic
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-05 13:56:57 +00:00
Jason A. Donenfeld
6fe055d0e8 Cleanup atomic getters
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-05 13:45:39 +00:00
Jason A. Donenfeld
2e7809f0d1 Version bump
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-05 10:58:04 +02:00
Jason A. Donenfeld
ee3254e46f Return with proper error status for bad address
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-05 08:49:07 +00:00
Jason A. Donenfeld
ef12ddb2be Do not take extra ActiveNBL when holding TransitionLock
The transition lock ensures that TunPause won't drop its last reference
until the shared transition lock is dropped.

Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-05 07:54:40 +00:00
Jason A. Donenfeld
34e61dc3b0 Do not complete pause if we're not running
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-05 07:43:17 +00:00
Jason A. Donenfeld
7437155446 Decrease alignment requirements to 4
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-05 06:15:46 +00:00
Jason A. Donenfeld
a0f031c692 Do not use _RESOURCES but rather allocate our own copy
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Fixes: 09dc932 ("Create copies of NBLs to complete them faster")
2019-07-05 06:15:46 +00:00
Simon Rozman
6e947a3813 Add _IRQL_requires_same_ support to clang formatting
Signed-off-by: Simon Rozman <simon@rozman.si>
2019-07-04 14:34:56 +02:00
Simon Rozman
db9eaf108e Declare TUN_MEMORY_TAG in network-byte order
...due to popular demand.

Signed-off-by: Simon Rozman <simon@rozman.si>
2019-07-04 14:34:56 +02:00
Simon Rozman
7ac5c57d41 Cleanup
Signed-off-by: Simon Rozman <simon@rozman.si>
2019-07-04 14:34:56 +02:00
Jason A. Donenfeld
f6663587da Version bump
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-04 14:34:56 +02:00
Jason A. Donenfeld
09dc9324d1 Create copies of NBLs to complete them faster
It turns out that waiting for them to come back in the return function
introduces extremely high latency.

We need to eventually stop doing this and move to a proper ring buffer.

Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-04 14:34:49 +02:00
Jason A. Donenfeld
b2182e0e90 Reorder ReturnNBL and DispatchWrite
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-03 08:50:30 +00:00
Jason A. Donenfeld
30d2b6e3a7 Receiving happens at dispatch level due to transition lock
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-03 08:50:30 +00:00
Jason A. Donenfeld
2517fb6659 Read userspace packet size once, before checks
Otherwise userspace can twiddle things between checks.

Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-03 08:50:30 +00:00
Jason A. Donenfeld
dbf234926d Use NDIS variable style for locals
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-03 08:50:30 +00:00
Jason A. Donenfeld
b71f64ae1c Use only multi-line comments
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-03 08:50:30 +00:00
Jason A. Donenfeld
1d96af3b98 Account for device removal before initialization
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-03 08:50:30 +00:00
Jason A. Donenfeld
248d4268df Use synchronize_rcu()-like semantics for exclusive transition lock
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-03 08:50:30 +00:00
Jason A. Donenfeld
33cac1114c Fix up comment about replacement for ->Reserved
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-03 08:50:30 +00:00
Jason A. Donenfeld
ae6e72a39e Simplify IRP processing after mapping
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-03 08:50:30 +00:00