Welcome Guest ( Log In | Register )

Outline · [ Standard ] · Linear+

 Most efficient way to share variable, across multiple threads (c#)

views
     
TSnarf03
post Dec 4 2023, 03:59 AM, updated 3y ago

Look at all my stars!!
*******
Senior Member
4,547 posts

Joined: Dec 2004
From: Metro Prima, Kuala Lumpur, Malaysia, Earth, Sol


Lets say i have a server program, it has 1 thread that has socket(s), and it is responsible to send data to all clients connected to this server. Then on the server it has many threads that can generate data, they need to "give" data to the socket thread so data can be distributed to all clients asap(cant wait for few seconds of data then only transmit once).

I wonder whats the most efficient way for all data generation threads to pass data to the socket thread ? Currently using "lock" and "list", all threads sharing a public variable, when any of the threads need to pass data, they will have to lock the variable, add/remove data, then unlock the variable, so other threads can access the same variable, if i have soo many threads, i think the variable will almost always in the "lock" status, are there any better way to do it ?


CODE

void main(){
 for (int i = 0; i < 100; i++) {
   Thread tAdd = new Thread(threadAdd);
   tAdd.Start();
 }
 Thread tRemove = new Thread();
 tRemove.Start();
}

Object lckData = new Object(); // for lock
List <string> lstData = new List<string>();

void threadAdd() {
 while (true) {
   lock(lckData) {  <--------------------------------- problem
     lstData.Add("New Data");
   }
   sleep(16); // 16 mili second = 1 frame
 }
}

void threadRemove() {
 while (true) {
   lock(lckData) {  <--------------------------------- problem
     while(lstData.Count > 0) {
       string strData = lstData[0];
       lstData.RemoveAt(0);
       fctDeliverData(strData);
     }
   }
   sleep(1);
 }
}

void fctDeliverData(string strData){
 // send data to clients, no issue
}


This post has been edited by narf03: Dec 4 2023, 03:41 PM
TSnarf03
post Dec 4 2023, 06:04 AM

Look at all my stars!!
*******
Senior Member
4,547 posts

Joined: Dec 2004
From: Metro Prima, Kuala Lumpur, Malaysia, Earth, Sol


QUOTE(jibpek @ Dec 4 2023, 05:42 AM)
Why still need to use Socket? That was the C/C++ era way. Now use RestFUL WebAPI will create it's own thread when client connect. Or you mean WebSocket?

Normally it is client to retrieve instead of server to send (proactively).
*
its a windows desktop application.
TSnarf03
post Dec 4 2023, 08:23 AM

Look at all my stars!!
*******
Senior Member
4,547 posts

Joined: Dec 2004
From: Metro Prima, Kuala Lumpur, Malaysia, Earth, Sol


QUOTE(jibpek @ Dec 4 2023, 07:47 AM)
Minimal API can run a RestFUL WebAPI server by itself. The size increases is < 2mb if I remember correctly.
*
i want to solve my problem, the problem isnt about moving away from c#, like somebody ask where to buy a soap, not expecting answer like you should stay next to a convenience shop. im sure your restful api has its advantage and disadvantage, i dont want to solve this problem then create another 10 problems after moving to restful api, unless you can ensure that there wont be any problem moving away from c# to restful api, restful is perfect, nobody face any problem.
TSnarf03
post Dec 4 2023, 08:58 AM

Look at all my stars!!
*******
Senior Member
4,547 posts

Joined: Dec 2004
From: Metro Prima, Kuala Lumpur, Malaysia, Earth, Sol


QUOTE(jibpek @ Dec 4 2023, 08:50 AM)
Minimal API is provided in .NET core. And it solved all problems associated with Socket communication.

Unless you are still working with .net fx 3.5
*
so whats your suggestion ? my problem isnt with socket, but sharing variables among threads. are u suggesting all threads make their own connections to client and send data instead of centralizing all data to 1 thread ? like 100 threads all send data individually to every single clients ?
TSnarf03
post Dec 4 2023, 09:42 AM

Look at all my stars!!
*******
Senior Member
4,547 posts

Joined: Dec 2004
From: Metro Prima, Kuala Lumpur, Malaysia, Earth, Sol


QUOTE(jibpek @ Dec 4 2023, 09:34 AM)
First define client and server correctly.

It is the Client initiate the connect to Server, not the other way around.

When a server accepted a connection from client, you can either:

- The socket way: Create a separate thread to handle it, otherwise it will block and no client can connect until your current connection is done.

- WebService / WebAPI: Will automatically create a separate thread for each client's connection

To handle the synchronization between threads:

- C++ socket way: PostThreadMessage

- MSMQ way

- C# way: Event

Your way can work as well. But you know the problem.
*
no, thats not the problem. my problem is like

server has communication thread, then it has data generation thread A B C D E F ..... AA AB AC AD .... CZ (can be 3 digits)

then there are client 1 2 3 4

every thread (A B C D E F ..... AA AB AC AD .... CZ) need to send data to EVERY CLIENT (1,2,3,4), currently all data goto comm thread, then comm thread send data to EVERY client, its not 1 to 1, its not many to many, its ALL to ALL.

i do not understand the purpose of your suggestion, i do not need alternative to socket unless it solve my problem, my problem is to solve overwhelming amount of threads need to submit data to 1 centralized thread and that thread also need to send data to every client, it has no trouble when the number is small, like 10-20 threads, but will start to have issue when the number get big.

This post has been edited by narf03: Dec 4 2023, 09:47 AM
TSnarf03
post Dec 4 2023, 10:06 AM

Look at all my stars!!
*******
Senior Member
4,547 posts

Joined: Dec 2004
From: Metro Prima, Kuala Lumpur, Malaysia, Earth, Sol


QUOTE(jibpek @ Dec 4 2023, 10:01 AM)
Your Server is the one that initiate the connection, and you insist on using Socket. Then you need to create a separate thread for each client (The client of the Socket in your Server). Pass data from your main thread to the client using Event instead of polling.

Last I checked, C# socket doesn't support async / await, if you do not wish to use thread, can use BeginSend, EndSend (kind of asynchronous programming)

Broadcasting to 10 ~ 20 devices is an easy job for today's computer. Even a raspberry pie running mono can easily communicate with 200+ devices without any problem if you engineered it correctly.
*
you still focus into the wrong thing ... nvm, i guess you have no intention to understand the problem.
TSnarf03
post Dec 4 2023, 03:40 PM

Look at all my stars!!
*******
Senior Member
4,547 posts

Joined: Dec 2004
From: Metro Prima, Kuala Lumpur, Malaysia, Earth, Sol


QUOTE(kingkingyyk @ Dec 4 2023, 10:55 AM)
Just use observer pattern. Instead of all threads throwing data to a list, you do :
- An event firer object and share it among all the threads
- Subscribe the socket send method (and the list of client you want to send to) to the event firer
- When data is emitted from any thread, call the subscribed socket method (pass the data as argument) in the event firer
- Perform lock/unlocking in your socket method to prevent clients receiving interleaved data.

So now your bottleneck will be just purely in your socket method, but you have more issues to solve later with this synchronized way. I.e. single client with extremely slow speed will cause other clients run into starvation since your socket send will just run forever.

For development or mock purpose this solution will be fine. I won't suggest something like this in production since it doesn't scale well and possible to get into DOS attack.

-----------------

The gold standard to solve this issue is by using message queue (doesn't matter the underlying is any of Websocket / AMQP / Stomp / MQTT)
*
why you guys keep thinking socket or communication is the bottleneck ? its not, why keep want to fix something that is not the problem ? look at the #1 post, i shown clearly where is the problem, its the lock

there are like 100 threads keep pushing data to 1 list, assuming 60 times per second, the lock will happen like 6000 times per second, if you think thats not the issue, then assume its 1000 threads, 60k unlock, 60k add data to the list per second, i want to solve THAT bottleneck.

 

Change to:
| Lo-Fi Version
0.0128sec    0.25    6 queries    GZIP Disabled
Time is now: 24th December 2025 - 07:31 AM