Commit Graph

262 Commits

Author SHA1 Message Date
Simon Rozman
14e5532dc8 Make receiving NBLs asynchronous
This commit moves NBL post-processing (moving ring head, releasing NBL)
to MINIPORT_RETURN_NET_BUFFER_LISTS handler.

Signed-off-by: Simon Rozman <simon@rozman.si>
2019-07-18 15:27:18 +02:00
Simon Rozman
f5eadb50c4 Minimize TransitionLock when receiving packets
We do not need to share-lock the TransitionLock for the whole life of
receiver thread.

Signed-off-by: Simon Rozman <simon@rozman.si>
2019-07-18 14:53:38 +02:00
Jason A. Donenfeld
9bf25d873b Fix insane coding style
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-18 12:49:32 +00:00
Jason A. Donenfeld
cdf0d0bf9a Fix awkward comment style
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-18 12:45:34 +00:00
Jason A. Donenfeld
5502b77218 Do not hijack PNP notifier
We no longer pend send-side NBLs, so we don't have a real use for this.

Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-18 12:41:04 +00:00
Jason A. Donenfeld
aac642e3dc Switch to paged dispatch handlers
We're not totally sure this is kosher, unfortunately.

Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-18 11:28:25 +00:00
Simon Rozman
043abc0a8e Minimize TransitionLock when sending packets
We do not need to share-lock the TransitionLock for complete NBL chain.
This commit should improve better state transition response, thou until
NDIS is sending a single NBL per MINIPORT_SEND_NET_BUFFER_LISTS call,
this should not have a considerable effect.

Since the skibNbl: call of NdisMSendNetBufferListsComplete() is made
inside the TransactionLock at dispatch IRQL, a dispatch IRQL hint was
added to the NdisMSendNetBufferListsComplete() call.

Signed-off-by: Simon Rozman <simon@rozman.si>
2019-07-18 13:52:16 +02:00
Jason A. Donenfeld
58ce3c5000 Piggy-back on top of NDIS' device object instead of adding our own
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-18 10:34:38 +00:00
Jason A. Donenfeld
bf51c91e3e README: clarify ring usage
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-17 21:13:39 +00:00
Simon Rozman
82ede2f96d Replace TUN_FLAGS_CONNECTED by an event
On client closing the handle there is a potential race if somebody
resets the TailMoved event of receive ring at the right time. Rather
than rely on TailMoved event, we introduce our own Disconnected
notification event and have the receive thread wait on both. The
Disconnected  event is also usable as TUN_FLAGS_CONNECTED substitute.

Signed-off-by: Simon Rozman <simon@rozman.si>
2019-07-17 14:53:09 +02:00
Simon Rozman
e7fad38a07 Improve lock retention when sending
NDIS may call MINIPORT_SEND_NET_BUFFER_LISTS from parallel threads to
queue as many packets as fast as possible. Initial implementation of
ring buffers used a spin lock to completely serialize sending packets
making it sub-optimal and burning large amount of CPU.

This commit uses locked section to allocate space for packet(s) in the
ring. It copies the packets unlocked, then it locks again to adjust the
ring tail.

Signed-off-by: Simon Rozman <simon@rozman.si>
2019-07-17 12:22:48 +02:00
Jason A. Donenfeld
1914547ab3 Spin for a bit before falling back to event object
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-17 10:17:46 +00:00
Simon Rozman
66e51bd08f Switch to ring buffers for exchanging packets
This demonstrates the use of ring buffers in its simplest, purest form.
No performance optimizations were made.

Signed-off-by: Simon Rozman <simon@rozman.si>
2019-07-16 20:06:54 +00:00
Jason A. Donenfeld
830df36cd3 README: correct path of device
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Reported-by: Shay Sharon <Shay.Sharon@audiocodes.com>
2019-07-09 11:10:17 +02:00
Simon Rozman
39939e3550 Rename remaining status -> Status and ctx -> Ctx
Signed-off-by: Simon Rozman <simon@rozman.si>
2019-07-08 11:10:43 +02:00
Jason A. Donenfeld
fb738767a0 Remove TunMapIrp
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-05 16:29:00 +00:00
Jason A. Donenfeld
17a0e2647a Simplify ActiveNblCount decrement
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-05 16:29:00 +00:00
Odd Stranne
50aa1f924c Rename NBL -> Nbl
Signed-off-by: Odd Stranne <odd@mullvad.net>
2019-07-05 16:28:59 +00:00
Odd Stranne
d268f03bc1 Rename labels to be more consistent
Signed-off-by: Odd Stranne <odd@mullvad.net>
2019-07-05 16:28:59 +00:00
Odd Stranne
75e434e3b8 Make error branching more compact
Signed-off-by: Odd Stranne <odd@mullvad.net>
2019-07-05 16:28:59 +00:00
Jason A. Donenfeld
20586e37b2 Treat ReferenceCount as an atomic
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-05 13:56:57 +00:00
Jason A. Donenfeld
6fe055d0e8 Cleanup atomic getters
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-05 13:45:39 +00:00
Jason A. Donenfeld
2e7809f0d1 Version bump
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-05 10:58:04 +02:00
Jason A. Donenfeld
ee3254e46f Return with proper error status for bad address
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-05 08:49:07 +00:00
Jason A. Donenfeld
ef12ddb2be Do not take extra ActiveNBL when holding TransitionLock
The transition lock ensures that TunPause won't drop its last reference
until the shared transition lock is dropped.

Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-05 07:54:40 +00:00
Jason A. Donenfeld
34e61dc3b0 Do not complete pause if we're not running
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-05 07:43:17 +00:00
Jason A. Donenfeld
7437155446 Decrease alignment requirements to 4
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-05 06:15:46 +00:00
Jason A. Donenfeld
a0f031c692 Do not use _RESOURCES but rather allocate our own copy
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Fixes: 09dc932 ("Create copies of NBLs to complete them faster")
2019-07-05 06:15:46 +00:00
Simon Rozman
6e947a3813 Add _IRQL_requires_same_ support to clang formatting
Signed-off-by: Simon Rozman <simon@rozman.si>
2019-07-04 14:34:56 +02:00
Simon Rozman
db9eaf108e Declare TUN_MEMORY_TAG in network-byte order
...due to popular demand.

Signed-off-by: Simon Rozman <simon@rozman.si>
2019-07-04 14:34:56 +02:00
Simon Rozman
7ac5c57d41 Cleanup
Signed-off-by: Simon Rozman <simon@rozman.si>
2019-07-04 14:34:56 +02:00
Jason A. Donenfeld
f6663587da Version bump
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-04 14:34:56 +02:00
Jason A. Donenfeld
09dc9324d1 Create copies of NBLs to complete them faster
It turns out that waiting for them to come back in the return function
introduces extremely high latency.

We need to eventually stop doing this and move to a proper ring buffer.

Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-04 14:34:49 +02:00
Jason A. Donenfeld
b2182e0e90 Reorder ReturnNBL and DispatchWrite
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-03 08:50:30 +00:00
Jason A. Donenfeld
30d2b6e3a7 Receiving happens at dispatch level due to transition lock
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-03 08:50:30 +00:00
Jason A. Donenfeld
2517fb6659 Read userspace packet size once, before checks
Otherwise userspace can twiddle things between checks.

Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-03 08:50:30 +00:00
Jason A. Donenfeld
dbf234926d Use NDIS variable style for locals
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-03 08:50:30 +00:00
Jason A. Donenfeld
b71f64ae1c Use only multi-line comments
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-03 08:50:30 +00:00
Jason A. Donenfeld
1d96af3b98 Account for device removal before initialization
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-03 08:50:30 +00:00
Jason A. Donenfeld
248d4268df Use synchronize_rcu()-like semantics for exclusive transition lock
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-03 08:50:30 +00:00
Jason A. Donenfeld
33cac1114c Fix up comment about replacement for ->Reserved
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-03 08:50:30 +00:00
Jason A. Donenfeld
ae6e72a39e Simplify IRP processing after mapping
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-03 08:50:30 +00:00
Jason A. Donenfeld
d82a68f830 Allow buffer mapping to happen concurrently
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-03 08:50:30 +00:00
Jason A. Donenfeld
88bde5b28e Correct and simplify page locking
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-03 08:50:30 +00:00
Jason A. Donenfeld
3d84bddcc0 Separate out MJ_CLOSE
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-03 08:50:30 +00:00
Jason A. Donenfeld
de481cdb12 Manually clean up ugly corners 2019-07-03 08:50:30 +00:00
Jason A. Donenfeld
5ec565c7e8 Improve designated initializers
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-03 08:50:30 +00:00
Jason A. Donenfeld
5bbff1026f Add NDIS team's clang-format conventions
This needs clang-format 9. This reveals a lot of other things we should
clean up.

Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Link: https://community.osr.com/discussion/291376/clang-format-and-driver-code
2019-07-03 08:50:30 +00:00
Jason A. Donenfeld
007ea09d1b Map user buffer only once
This avoids needless page table modifications and also lets us enforce
having writable pages.

Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-03 08:50:30 +00:00
Jason A. Donenfeld
d50cab5732 Consider receive NBLs to be immutable
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2019-07-03 08:50:30 +00:00