GNN social graph: where link prediction breaks
A model on a social graph doesn't make mistakes randomly: failures cluster in specific areas. Link quality there degrades predictably — by node degree, community growth rate, and how old the labels are.
Therefore, an averaged metric over the entire dataset says nothing about how the solution will behave on new users.
Rare links are the first failure. For a node with degree one or two, neighbor aggregation turns noise into the basis of prediction: there is less signal than variability.
The model either underestimates such pairs or pulls them toward the population average — and these are exactly the ones that most often turn out to be target pairs.
The second area is bot labeling. Automated accounts create dense cliques with nearly identical features, and the model quickly learns them as a strong link signal. Then link prediction reproduces bot activity as the norm, while genuine weak links drop down in ranking.
Community detection breaks under rapid growth: in a couple of weeks, the group's composition changes so much that the learned representations describe a structure that no longer exists.
Added to this is outdated labeling — links marked as significant have long since dissolved, and no one has marked new ones.
We break down the graph by these zones separately, not with a single number across the entire dataset: noise in a social graph must first be localized, otherwise any fine-tuned model simply transfers the error to another area. Below is how this works.
What changes after implementation: quality, speed, failures
Average accuracy over the graph is deceptive: it rests on popular nodes, while decisions are made based on rare links.
Therefore, we calculate the quality metric separately on the tail of the distribution — the share of correct predictions for pairs with one or two common links, and that is exactly where the model provides the main gain.
We train on graph neural networks in PyTorch Geometric: signal from second- and third-order neighbors works where frequency features are silent.
Link freshness is the second metric. Instead of fully recomputing embeddings across the entire graph, we compute incrementally: we update affected nodes and their neighborhood, while a full run is left on a schedule to clean out accumulated drift. A new edge enters the model within a short window, not by the next export.
We catch manipulation on structure, not on counters: dense cliques of fresh accounts, anomalous degree growth, matching link patterns. Mass following changes the numbers, but distorts local topology — this is visible before the scheme has had time to trigger.
The model outputs a confidence score, not a label, so only the borderline zone goes to manual review — the share of reviews decreases, and disputed cases remain with the analyst.
Labels from reviews are returned to training, and the next recomputation removes part of the load from people.
A separate class of failures is gone, where the graph was built from a snapshot and the model decided on yesterday's links: drift control shows degradation in recall and precision before it reaches decisions.
Comparison of approaches: rules, embeddings, link-based models
The choice between approaches is not about the "strongest" solution, but about what gives acceptable accuracy on your data and survives the next change in audience behavior.
At the start, the task is almost always attempted to be closed with rules: activity over a window, share of common neighbors, threshold by number of messages.
Rules and heuristics are readable, explainable, and work without labels — but each new rule conflicts with the previous one, and thresholds have to be re-tuned with every shift in activity.
| Approach | When it fits | Limitations |
|---|---|---|
| Hard rules and heuristics | Need a quick explainable result, no labels | Number of rules grows faster than accuracy, false positives accumulate |
| Statistical node features | Have profile aggregates: activity, link composition | Ignores structure — node evaluation does not depend on the neighborhood |
| Vector node representations | Need to compare nodes and find similar ones | Collapses context: representation does not change with neighbors |
| Link-based model | Paths, communities, and link prediction matter | Requires data and labels, more expensive to support and explain the decision |
Statistical features and vector node representations occupy the middle: the former are cheap and predictable, the latter provide comparison of similar participants, but both schemes look at a node separately from the graph.
A link-based model reads the neighborhood as a whole and therefore works better where the nature of the links matters.
In practice, the choice of approach most often turns out to be a cascade: rules cut off obvious cases, vector representations provide candidates, and the link-based model ranks them by probability.
Community clustering adds a group level — a bot farm is visible not by an individual node, but by dense connectivity between clusters that should be almost unconnected.
How we build the model: stages of engineering work
We carry out the development of a graph neural network (GNN) on a social graph as a chain of steps, where each relies on the fixed result of the previous one.
The order is fundamental: if features and data splitting are assembled "by eye", the model shows beautiful metrics on training and falls apart on live data. Therefore, we fix the schema and rules before starting training.
- Data audit. We look at what links and attributes exist in the graph: follows, interaction history, bot and community labels. We check duplicates, gaps, and the boundary from which data is suitable for training.
- Feature schema. We describe what is fed to the node and edge: degree, account age, overlap of neighborhoods, activity in communities. The schema determines what data to collect next.
- Training set assembly. We split history by time, not randomly, otherwise the model will peek into the future. We fix labels at the time of splitting and take negative examples from pairs with no link.
- Model tuning. We select the neighbor traversal depth, the share of dropped links, and rules for rare nodes. Separately, we check whether there is overfitting on the most active participants.
- Validation on a holdout period. We calculate the share of correct link predictions, recall on bots, and stability by communities on a segment the model has not seen.
- Solution handover. We deliver the model, feature preparation code, the fixed schema, and recomputation instructions — the client's team runs everything themselves.
- Update policy. The social graph changes daily: we set the retraining period, control metrics, and rules for rolling back to the previous version.
Implementation artifacts: schemas, configurations, policies
After implementation, the client retains not only a working model but also engineering artifacts: with them, the solution is developed in-house or handed over to another team without reverse engineering. Each document is tied to a specific graph processing step and is updated together with the model.
- Data processing schemas. The path from sources to the graph: how vertices and edges are formed, where identifiers are normalized and duplicates are cut off, which attributes go into features. Connecting a new source is an edit to one node of the schema.
- Model configurations. Neighbor aggregation layers, neighborhood size, class weights, time-based sample splitting. Random splitting on a graph with temporal links inflates metrics and does not transfer to production.
- Labeling rules. What counts as a bot versus an active participant, how suspicious links and communities are marked, how annotator agreement is checked. Otherwise, the model learns from contradictions.
- Access policies. Who has access to the full graph, who only to aggregates, which fields are anonymized at input, who can export a subgraph. The request log closes security service questions.
- Retraining policy. Triggers — drift in graph structure, metrics falling below threshold, emergence of a new link type; frequency, validation on a holdout period, procedure for rolling back to the previous version.
- Documentation and reproducibility. Feature map, metric dictionary with target values, instructions for launching training and outputting predictions. The process is reproduced from scratch on clean data.
Case study: bot labeling in a community, metrics before and after
Before
A community in Discord grew to tens of thousands of accounts, and manual labeling could no longer keep up: two moderators checked complaints one profile at a time, while bots arrived in clusters. While they dealt with one, several more appeared — the queue grew faster than it was processed.
Accuracy on rare links sagged: a single bot is almost indistinguishable from a newcomer, while a cluster gives itself away by its structure. They blocked the active ones — the core of the cluster remained and restored the network within a few days.
What we changed
We moved labeling from individual accounts to an interaction graph: nodes are profiles and channels, edges are replies, reactions, and co-occurrences in threads.
The link-based model propagates features across the neighborhood and labels the entire cluster, so weak signals from one profile are reinforced by signals from neighbors. Rare but recurring pairs were given a separate weight.
Labeling accuracy grew from 0.62 to 0.89, recall on rare links — from 0.41 to 0.78. The share of clusters found entirely before a spike rose from a third to 85%, and the manual review queue shrank fourfold: the moderator confirms a decision by cluster, not by each profile.
Community dynamics changed: the share of re-registrations after a block fell by about three times — the core of the cluster leaves along with the active ones.
The model only ranks candidates; the final decision remains with the moderator, so rolling back to manual labeling does not break the process.
What to do if the graph is already built but link quality is low
Poor quality on a GNN is rarely explained by architecture — in most cases the cause is in the data.
Before changing the aggregation depth, three things are ruled out: leakage of target edges into features, noise in labeling, and desynchronization of training and validation samples by time.
Diagnostics proceed from symptom to cause:
| Symptom | Likely cause | First check |
|---|---|---|
| Metrics rise on training, fall on validation | Overfitting to structure | Cut traversal depth, add edge regularization |
| Quality gradually degrades over time | Distribution shift | Look at feature drift and new link types by windows |
| New participants are predicted worse than old ones | Node cold start | Calculate the share of nodes with degree less than two |
| Model is confidently wrong | Noise in labeling | Recheck disputed edges and annotator agreement |
The order of diagnostics is the reverse of the usual one — from data to model.
First, check labeling on a subsample of links and agreement between annotators, then metrics broken down by time windows, and only then architecture: aggregation depth, class weights, negative example selection.
Libraries like PyTorch Geometric provide ready-made layers and graph loaders, but the sequence of checks remains with the engineer.
Cold start is addressed separately. For nodes with degree less than two, features are taken from the profile rather than the neighborhood, and such nodes are masked in the loss function so as not to blur the gradient.
Distribution shift is caught by a sliding retraining window — when the share of new link types exceeds the threshold, the model is retrained from scratch rather than fine-tuned on outdated data.
A sign that the order worked: the gap between training and validation stops growing, and quality on newbie nodes catches up to the graph average.
Doubts about implementation: scale, labeling, privacy
Doubts at the outset are almost always the same: there is little or too much data, labeling is expensive, privacy is in question, and the result is not needed immediately. Let's go through five technical questions that determine the work plan.
How many links will have to be labeled manually?
Fewer than it seems: the model draws its main signal from graph structure, not from manual labels. We manually label only the validation set and disputed cases — hundreds of links, balanced by node types.
What happens when the graph grows by an order of magnitude?
Training does not run on the entire graph but on node neighborhoods: the data volume grows, not the computation time. The model is fine-tuned incrementally, and storage is separated from computation.
How does this combine with privacy?
The model sees anonymized identifiers and link topology; personal attributes do not enter training. Aggregates, if necessary, are computed inside your perimeter, and only the result goes outside.
Can it be implemented in stages?
Yes, and this is the preferred path. First, a check on accumulated history, then shadow mode — predictions are computed but decisions are made by the current logic — and only then a limited perimeter.
How can we be sure the predictions can be trusted?
We look not only at quality metrics but also at confidence calibration; we manually check disputed links. We set the threshold so that the model stays silent on ambiguous links, and rolling back to rules remains a single command.
Let's discuss the task: what to send for an initial assessment
To come back with a timeline estimate and work plan rather than general words, a few initial data points are enough.
There is no need to prepare a full technical specification at this stage — the scope of work is estimated from a description of the scenario and one real graph fragment.
- Application scenario: who receives the result and what decision they make — bot blocking, link recommendation, or community moderation.
- Example export: an anonymized graph fragment — vertices, edges, link types and attributes, volume and observation period.
- Current rules: the heuristics and thresholds by which decisions are made now, and where they fail.
- Quality metric: what is considered a correct answer and by what share of correct hits you evaluate the current solution.
- Labels: whether there are confirmed examples — known bots, confirmed links, labeled communities.
- Constraints: requirements for privacy, retention periods, access, and data anonymization.
- Expected effect: what should change after implementation and in what numbers you measure it.
From this set, the problem statement takes shape: we select features for vertices and links, plan validation on a holdout period, and fix the acceptance metric. Describe the scenario in free form — we will come back with a timeline estimate and a work plan by stages.
