[{"content":"This was not a 100-day streak. The dates make that clear. Day 26 was published on September 30, 2021. Day 66 arrived on January 20, 2023. Day 77 followed on March 14, 2024, Day 91 on August 15, 2025, and Day 99 on February 13, 2026.\nI stopped and restarted several times. The questions changed with me. What began as a way to learn more Java became a record of how I investigate runtime behavior.\nAt the beginning, that mostly meant learning more APIs, more features, more idioms, and more little utilities I could reach for later. Some of the early posts are exactly that. They are small, direct, and useful in a narrow way: retrying a method with TimerTask, emulating a Pair, converting an Iterator into a Spliterator, working through individual language features one by one.\nI still value those posts. They show me learning in public in the most literal sense.\nBut when I look across the posts now, that is not the real story it tells. The bigger change is that the challenge slowly stopped being about collecting Java knowledge and started becoming a way to investigate behavior. I became less interested in naming abstractions and more interested in asking what they were hiding. Memory leaks. Heap dumps. JMX. Off-heap memory. Event loops. Virtual threads. Work distribution. Startup time. The center of gravity moved from \u0026ldquo;what does this API do?\u0026rdquo; to \u0026ldquo;why is this system behaving like that?\u0026rdquo;\nThe writing changed with that shift. Early on, I was still trying to sound like I understood the topic before I really did. Somewhere along the way, I stopped trying to sound authoritative and got more comfortable writing from confusion. That made the posts less polished in one sense, but more honest in a better one. The later posts that stayed with me usually begin with a wrong assumption, a gap in my mental model, or a question I could not shake. That is much closer to how I actually learn.\nIf I had to reduce the whole challenge to one sentence, it would be this: it started as learning Java and gradually became learning how systems behave.\nFrom Explaining Topics to Investigating Behavior One of the clearest differences between the earlier and later parts of these posts is not the topic list. It is the shape of the questions.\nThe earlier posts often behave like explainers. I pick a concept, define it, show a code snippet, and move on. That mode was useful because I was building vocabulary and touching a lot of surface area. It gave me repetition. It forced me to keep shipping. It also sometimes let me stay a little too comfortable. I could finish a post having explained a tool without really testing the edges of my own understanding.\nThe startup-time rabbit hole in Day 94 starts from a concrete problem and then keeps asking where startup time actually goes. It is less \u0026ldquo;here are some optimizations\u0026rdquo; and more \u0026ldquo;I had been thinking about startup time the wrong way.\u0026rdquo; That difference matters.\nBy Day 98, the frame is even clearer. The post is not built around \u0026ldquo;here is virtual threads.\u0026rdquo; It is built around a mistaken model: I thought virtual threads were just green threads, and that model stopped explaining what I was seeing. The writing gets better there because the question gets better.\nThen Day 99 does the same thing again. If virtual threads make blocking I/O scale, why do event loops still exist? That is a much more alive question than a generic comparison between two frameworks. It starts from a misconception, follows it down into selectors and multiplexing and then I understood that its a trade-off\nEven some of the posts I would now write differently were moving in that direction. The one about startup time, the one about printing and logging cost, the later concurrency series, the memory-mapping deep dive, all of them have the same underlying motion: they begin with something that I felt obvious from the surface and then stop being obvious as soon as I looked under it.\nMechanisms Started Mattering More Than Frameworks The other change is even simpler to name because I can feel it in the topics themselves: I became less interested in frameworks and more interested in systems and how are they working to make the things tha I want.\nThe memory leak series around Day 66, the heap dump post, the JMX posts, and the OutOfMemory alert post all pulled in the same direction. They made memory stop feeling like a mysterious background concern and start feeling like something I could inspect. Retained references, heap state, threshold notifications, dump files, leak patterns, object lifetime: those are runtime-behavior topics, not framework topics.\nThen the later memory-heavy posts pushed that further. In Day 77, I was writing to allocated memory through Unsafe. The next step was the safer FFM direction. By Day 91, I was looking at memory mapping not as a convenient API trick but as a question about where data actually lives, when it gets loaded, and how much of the work belongs to the heap versus the operating system.\nEven the JMH posts and the later performance posts helped here. They moved performance a little farther away from opinion and a little closer to measurement. The printing post is not really about println() versus file writing. It is about where overhead comes from and what kind of cost a friendly abstraction is hiding.That does not mean frameworks stopped mattering. They just stopped feeling sufficient as explanations.\nDay 95 and the two posts that follow it are broad on the surface. They cover concurrency tools. But even there, the interesting part is not \u0026ldquo;here are twelve APIs.\u0026rdquo; The interesting part is the repeated warning that concurrency is not automatically a win, that the wrong synchronizer creates the wrong shape of failure, and that production systems need coordination as much as they need execution.\nThat theme becomes much more concrete in those later virtual-thread and event-loop posts. Once I started looking at virtual threads as continuations, heap-allocated stack chunks, mount and unmount behavior, pinning, carrier threads, selectors, and event loops, the old question \u0026ldquo;which framework should I use?\u0026rdquo;. The new question I started to ask was what is happening inside and how can it help.\nThe spliterator material that I thought I would write for 100th day makes more sense to me as one late example of that shift than as the final centerpiece. The real lesson is not that I wrote a custom Spliterator. It is that .parallel() hid a work-distribution policy I had not thought about carefully enough. Once I noticed the growing-batch behavior, the problem stopped being \u0026ldquo;why is Java Streams slow?\u0026rdquo; and became \u0026ldquo;how is work being divided, why are cores going idle, and what happens when the default coordination strategy does not fit the workload?\u0026rdquo;\nAt the code level, the turning point was almost embarrassingly small:\n1 2 3 4 5 6 7 8 9 10 @Override public Spliterator\u0026lt;T\u0026gt; trySplit() { final HoldingConsumer\u0026lt;T\u0026gt; holder = new HoldingConsumer\u0026lt;\u0026gt;(); if (!tryAdvance(holder)) return null; final Object[] batch = new Object[batchSize]; int j = 0; do { batch[j] = holder.value; } while (++j \u0026lt; batchSize \u0026amp;\u0026amp; tryAdvance(holder)); return Spliterators.spliterator(batch, 0, j, characteristics()); } The code itself is not the point. The point is that the real problem was sitting one layer below the API I thought I was using. That is basically the whole point. I kept starting from an abstraction I already knew how to use. Then I ran into behavior that abstraction did not explain. Then I followed the behavior down a level.\nWhat the Writing Habit Changed This #100DaysOfJava challange helped me to better think by writing these online. Publishing regularly forced vague curiosity to become a good question. It is easy to carry around a fuzzy feeling that something is intriguing. It is harder to turn that into a post that begins somewhere concrete, moves through evidence, and ends with a claim that is modest enough to defend.\nI thought virtual threads were just green threads. I thought virtual threads might make event loops obsolete. I saw memory mapping as a magic trick and then tried to understand the path from kernel to code. I treated startup time like a vague Java problem and then tried to locate the actual bottlenecks.\nThat pattern is closer to how I really learn than the older style where I tried to sound finished too early. Writing from confusion did not make me less rigorous. It made the missing rigor easier to locate. If I can name the broken model, I know what I am testing.\nThat is probably the main writing lesson I want to keep. Not \u0026ldquo;publish every day.\u0026rdquo; Not \u0026ldquo;build in public.\u0026rdquo; Just this: when a question is still blurry, stay with it until its a little bit clearer.\nWhat Changed In How I Think About Software Engineering I did learn a littl bit more Java from this challenge.I learned APIs I never had to work with directly in my day job, internals which I didnt new how it worked or even existed, language features I had only used casually, and tools I probably would not have opened if I havent written these series.\nThis is not mastery. If anything, it has made me more aware of how much I still do not understand. But it is a better kind of ignorance. It gives me better questions.\nThat is also why Chaos;Code;Clarity my new newsletter where I will explore the chaotically beautiful world of Software Engineering and AI. This feels like a natural next step to me. Not because I am done with Java, and not because this challenge ended in some clean graduation. It feels natural because these posts was already moving there. The part I want to carry forward is the habit underneath it: think, probe, measure, question the default story, and keep following the system until the behavior becomes a little less mysterious.\nI started this project to learn more Java. I am closing the numbered series with a better method: begin with a concrete question, inspect the system, measure what changes, and state what the evidence cannot prove.\nLimitations The archive spans several years, JDK versions, and levels of rigor. Early posts are short learning notes, while later posts include larger experiments. Treat each article according to the versions, commands, and evidence it records. The series is a learning history, not one controlled study.\n","permalink":"https://mohibul.pages.dev/posts/posts/java/100daysofjava/day100-capstone.md/","summary":"This challenge started as a way to learn more Java. Over time, it became a way to follow confusion down to memory, coordination, performance, and failure modes.","title":"What 100DaysOfJava changed in how I investigate systems"},{"content":"After Day 98, I wanted to test a narrower question: when does a virtual-thread server behave differently from an event-loop server?\nI built one server for each model and published the code and benchmark scripts. The comparison does not establish a universal winner. It shows how the models behave under one recorded workload and makes that workload repeatable.\nTest context Repository: virtual-thread-eventloop-test Runtime requirement: JDK 21 or later Recorded machine: Windows, 16 CPU cores, 31.82 GB RAM, and 183 GB free disk Server heap: 4 GB for each implementation Load generator: Bombardier with 14 worker threads Commands and raw-analysis steps: benchmarks/README.md and benchmarks/QUICK-REFERENCE.md in the repository The recorded results are a sample run, not a production load test. Hardware, network topology, request mix, warmup, heap size, and connection behavior can change the outcome.\nWhy compare the models Virtual threads make blocking I/O practical at high concurrency while keeping sequential control flow. Event loops use a smaller number of threads and explicit state machines to manage many connections. Both models remain useful because they optimize different constraints.\nVirtual threads can unmount from a carrier thread during supported blocking operations, allowing that carrier to run other work. This keeps sequential control flow practical for request handlers that wait on databases, remote APIs, or files.\nBut reactive frameworks don\u0026rsquo;t work this way. They use event loops: one thread handles thousands of connections by multiplexing I/O events. No mounting. No unmounting. No stack switching. Just a tight loop reading from a Selector.\nI needed to understand both models to know when each wins.\nNon-Blocking I/O: The Foundation Platform-thread-per-request designs can spend substantial resources on threads that are waiting. Virtual threads reduce that cost, but their stack chunks and application state still consume memory, and scheduling is not free. Measure those costs under the intended concurrency and workload.\nNon-blocking I/O takes a different approach: one thread, many connections, explicit multiplexing.\nWhat is I/O multiplexing I/O multiplexing breaks down to kernel-level efficiency: one thread polls multiple file descriptors via system calls like select()/poll()/epoll(), reacting only to ready I/O events to avoid per-connection blocking.\nAt the OS kernel level, I/O operations cross between user space and kernel space. With a platform-thread-per-connection design, a blocking socket.read() leaves that thread waiting for data. Large connection counts can therefore make thread memory and scheduling part of the system\u0026rsquo;s resource budget.\nMultiplexing inverts this model. Instead of one thread per connection, one thread monitors many connections. The kernel tells you which connections are ready for I/O, and you react only to those.\nHere\u0026rsquo;s how it works at the kernel level:\nThe select() system call (or epoll on Linux, kqueue on macOS) takes a set of file descriptors with interest operations (read/write/accept), atomically blocks until any file descriptor signals readiness via kernel events, then returns a bitmask of ready file descriptors all without per-fd polling.\nHow It Works In Java Java NIO\u0026rsquo;s Selector wraps this mechanism. On Linux, the JDK can use an epoll-backed selector. The selector maintains registered channels and their interest operations. When you call selector.select(), it waits until at least one channel is ready, then returns the ready keys.\nNon-blocking channels ensure read()/write() return immediately they never block. If data isn\u0026rsquo;t ready, read() returns 0 bytes. If the socket buffer is full, write() returns 0 bytes written. This forces applications to re-check readiness via selector keys in the event loop.\nByteBuffer manages data with position/limit/capacity semantics. After reading, you call flip() to prepare for consumption: it sets limit = position and position = 0. This is critical without flip(), you\u0026rsquo;ll read from the wrong position or read garbage data.\nThe Reactor Pattern layers on top of multiplexing. It consists of:\nAcceptor: Handles new connections, registers clients with the selector Demultiplexer: The Selector.select() call that waits for events Dispatcher: Routes SelectionKey events to appropriate handlers Handler: Business logic that processes the I/O event Java frameworks such as Netty and Reactor Netty build on event-loop designs. A Reactor Netty TcpServer, for example, uses Netty event-loop groups and exposes work through reactive streams. The achievable connection count depends on buffers, application state, traffic, operating-system limits, and hardware.\nA single-thread event loop processes sequentially: select → dispatch → callback.\nIf a handler blocks, it stalls the entire loop that\u0026rsquo;s why reactive frameworks emphasize non-blocking handlers.\nThe key lifecycle per channel:\nRegister interest operations (e.g., OP_ACCEPT for server sockets, OP_READ for client sockets) select() yields a set of ready operations (readyOps) Process the event (read → flip buffer → handle → set OP_WRITE if partial write) Cancel the key after use to avoid duplicate events Here\u0026rsquo;s a minimal example showing the pattern:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 Selector selector = Selector.open(); ServerSocketChannel server = ServerSocketChannel.open(); server.configureBlocking(false); server.bind(new InetSocketAddress(8080)); server.register(selector, SelectionKey.OP_ACCEPT); while (true) { selector.select(); // Blocks until at least one channel is ready for (SelectionKey key : selector.selectedKeys()) { if (key.isAcceptable()) { SocketChannel client = server.accept(); client.configureBlocking(false); client.register(selector, SelectionKey.OP_READ); // Interest set } else if (key.isReadable()) { SocketChannel client = (SocketChannel) key.channel(); ByteBuffer buf = ByteBuffer.allocate(1024); buf.clear(); int bytes = client.read(buf); if (bytes \u0026gt; 0) { buf.flip(); // Prepare for reading // Process buf.array()[0..bytes] key.interestOps(SelectionKey.OP_WRITE); // Switch to write mode } } else if (key.isWritable()) { SocketChannel client = (SocketChannel) key.channel(); ByteBuffer buf = (ByteBuffer) key.attachment(); client.write(buf); if (!buf.hasRemaining()) { key.interestOps(SelectionKey.OP_READ); // Switch back to read mode } } selector.selectedKeys().remove(key); // Must remove to avoid reprocessing } } This ping-pongs data efficiently. For production systems, you\u0026rsquo;d add write queues (only register OP_WRITE when the queue is non-empty) and handle partial reads/writes properly. See in the below diagram how this event loop works.\nThe selector waits when no registered I/O is ready. When a channel becomes ready, the loop processes that event without assigning a dedicated platform thread to every idle connection. The design still has scheduling, system-call, buffer, and handler costs; it changes where those costs appear.\nHere\u0026rsquo;s the core pattern using Java NIO:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 import java.io.IOException; import java.net.InetSocketAddress; import java.nio.ByteBuffer; import java.nio.channels.SelectionKey; import java.nio.channels.Selector; import java.nio.channels.ServerSocketChannel; import java.nio.channels.SocketChannel; import java.util.Iterator; import java.util.Set; public class NonBlockingServer { private static final int PORT = 8080; private static final int BUFFER_SIZE = 1024; public static void main(String[] args) throws IOException { // Open selector - the heart of event multiplexing Selector selector = Selector.open(); // Create server socket, make it non-blocking ServerSocketChannel serverSocket = ServerSocketChannel.open(); serverSocket.bind(new InetSocketAddress(PORT)); serverSocket.configureBlocking(false); // Register interest in accept events serverSocket.register(selector, SelectionKey.OP_ACCEPT); System.out.println(\u0026#34;Non-blocking server started on port \u0026#34; + PORT); ByteBuffer buffer = ByteBuffer.allocate(BUFFER_SIZE); while (true) { // Block until events are ready (but only this ONE thread blocks) selector.select(); // Get all ready events Set\u0026lt;SelectionKey\u0026gt; selectedKeys = selector.selectedKeys(); Iterator\u0026lt;SelectionKey\u0026gt; iter = selectedKeys.iterator(); while (iter.hasNext()) { SelectionKey key = iter.next(); iter.remove(); if (!key.isValid()) { continue; } if (key.isAcceptable()) { handleAccept(key, selector); } else if (key.isReadable()) { handleRead(key, buffer); } else if (key.isWritable()) { handleWrite(key); } } } } private static void handleAccept(SelectionKey key, Selector selector) throws IOException { ServerSocketChannel serverSocket = (ServerSocketChannel) key.channel(); SocketChannel client = serverSocket.accept(); client.configureBlocking(false); // Register interest in read events for this client client.register(selector, SelectionKey.OP_READ); System.out.println(\u0026#34;Accepted connection from \u0026#34; + client.getRemoteAddress()); } private static void handleRead(SelectionKey key, ByteBuffer buffer) throws IOException { SocketChannel client = (SocketChannel) key.channel(); buffer.clear(); int bytesRead; try { bytesRead = client.read(buffer); } catch (IOException e) { closeConnection(key); return; } if (bytesRead == -1) { closeConnection(key); return; } // Store the read data for writing back buffer.flip(); key.attach(buffer.duplicate()); key.interestOps(SelectionKey.OP_WRITE); } private static void handleWrite(SelectionKey key) throws IOException { SocketChannel client = (SocketChannel) key.channel(); ByteBuffer buffer = (ByteBuffer) key.attachment(); client.write(buffer); if (!buffer.hasRemaining()) { // Done writing, switch back to reading key.interestOps(SelectionKey.OP_READ); key.attach(null); } } private static void closeConnection(SelectionKey key) throws IOException { key.cancel(); key.channel().close(); } } This is the foundation. One thread handles all connections. The Selector monitors multiple channels. When data arrives, the selector wakes up with ready events. We handle them without blocking.\nKey insight: selector.select() is the only blocking call. Everything else accept(), read(), write() returns immediately. If data isn\u0026rsquo;t ready, the operation returns zero bytes. No waiting.\nBuilding an Event Loop HTTP Server The pattern above is raw NIO. Let\u0026rsquo;s build something more real: an HTTP server using the event loop pattern.\nEvent loop = infinite loop + selector + event handlers + state machines.\nHere\u0026rsquo;s an instructional implementation for the comparison:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 package org.example; import java.io.IOException; import java.net.InetSocketAddress; import java.nio.ByteBuffer; import java.nio.channels.*; import java.nio.charset.StandardCharsets; import java.util.*; import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.atomic.AtomicLong; public class EventLoopHttpServer { private static final int PORT = 8081; // Different port private static final int BUFFER_SIZE = 8192; private final Selector selector; private final ServerSocketChannel serverSocket; private final Map\u0026lt;SocketChannel, ConnectionState\u0026gt; connections; // Metrics private final AtomicLong totalRequests = new AtomicLong(0); private final AtomicLong activeConnections = new AtomicLong(0); public EventLoopHttpServer() throws IOException { this.selector = Selector.open(); this.serverSocket = ServerSocketChannel.open(); this.connections = new ConcurrentHashMap\u0026lt;\u0026gt;(); serverSocket.bind(new InetSocketAddress(PORT)); serverSocket.configureBlocking(false); serverSocket.register(selector, SelectionKey.OP_ACCEPT); } public void start() throws IOException { System.out.println(\u0026#34;Event Loop HTTP Server started on port \u0026#34; + PORT); System.out.println(\u0026#34;Thread: \u0026#34; + Thread.currentThread().getName()); System.out.println(\u0026#34;Available processors: \u0026#34; + Runtime.getRuntime().availableProcessors()); // Print stats every 10 seconds long lastStatTime = System.currentTimeMillis(); while (true) { selector.select(1000); // Timeout for stats printing long now = System.currentTimeMillis(); if (now - lastStatTime \u0026gt; 10_000) { System.out.printf(\u0026#34;Stats - Total requests: %d, Active connections: %d, Thread: %s%n\u0026#34;, totalRequests.get(), activeConnections.get(), Thread.currentThread().getName()); lastStatTime = now; } Iterator\u0026lt;SelectionKey\u0026gt; iter = selector.selectedKeys().iterator(); while (iter.hasNext()) { SelectionKey key = iter.next(); iter.remove(); if (!key.isValid()) { continue; } try { if (key.isAcceptable()) { handleAccept(); } else if (key.isReadable()) { handleRead(key); } else if (key.isWritable()) { handleWrite(key); } } catch (IOException e) { closeConnection(key); } } } } private void handleAccept() throws IOException { SocketChannel client = serverSocket.accept(); if (client == null) { return; } client.configureBlocking(false); SelectionKey key = client.register(selector, SelectionKey.OP_READ); ConnectionState state = new ConnectionState(); connections.put(client, state); activeConnections.incrementAndGet(); key.attach(state); } private void handleRead(SelectionKey key) throws IOException { SocketChannel client = (SocketChannel) key.channel(); ConnectionState state = (ConnectionState) key.attachment(); state.readBuffer.clear(); int bytesRead; try { bytesRead = client.read(state.readBuffer); } catch (IOException e) { closeConnection(key); return; } if (bytesRead == -1) { closeConnection(key); return; } if (bytesRead == 0) { return; } // Check if we\u0026#39;ve read a complete HTTP request state.readBuffer.flip(); byte[] data = new byte[state.readBuffer.remaining()]; state.readBuffer.get(data); String request = new String(data, StandardCharsets.UTF_8); if (request.contains(\u0026#34;\\r\\n\\r\\n\u0026#34;)) { // Complete request received totalRequests.incrementAndGet(); // Simulate work (like the virtual thread version) simulateWork(); prepareResponse(state, request); key.interestOps(SelectionKey.OP_WRITE); } else { // Incomplete request, keep reading state.readBuffer.compact(); } } private void simulateWork() { // Simulate work without blocking the event loop // In reality, you\u0026#39;d use async I/O or offload to thread pool // For benchmark parity, we\u0026#39;ll do a tiny computation long start = System.nanoTime(); while (System.nanoTime() - start \u0026lt; 10_000_000) { // Busy-wait for 10ms (simulates non-blocking work) // This is NOT how you\u0026#39;d do it in production, but matches the virtual thread version } } private void prepareResponse(ConnectionState state, String request) { // Parse request line String[] lines = request.split(\u0026#34;\\r\\n\u0026#34;); String requestLine = lines.length \u0026gt; 0 ? lines[0] : \u0026#34;UNKNOWN\u0026#34;; String[] parts = requestLine.split(\u0026#34; \u0026#34;); String method = parts.length \u0026gt; 0 ? parts[0] : \u0026#34;UNKNOWN\u0026#34;; String path = parts.length \u0026gt; 1 ? parts[1] : \u0026#34;/\u0026#34;; // Build response String body = String.format( \u0026#34;{\\\u0026#34;message\\\u0026#34;:\\\u0026#34;Hello from Event Loop\\\u0026#34;,\\\u0026#34;method\\\u0026#34;:\\\u0026#34;%s\\\u0026#34;,\\\u0026#34;path\\\u0026#34;:\\\u0026#34;%s\\\u0026#34;,\\\u0026#34;thread\\\u0026#34;:\\\u0026#34;%s\\\u0026#34;,\\\u0026#34;timestamp\\\u0026#34;:%d}\u0026#34;, method, path, Thread.currentThread().getName(), System.currentTimeMillis() ); String response = \u0026#34;HTTP/1.1 200 OK\\r\\n\u0026#34; + \u0026#34;Content-Type: application/json\\r\\n\u0026#34; + \u0026#34;Content-Length: \u0026#34; + body.length() + \u0026#34;\\r\\n\u0026#34; + \u0026#34;Connection: keep-alive\\r\\n\u0026#34; + \u0026#34;\\r\\n\u0026#34; + body; state.writeBuffer = ByteBuffer.wrap(response.getBytes(StandardCharsets.UTF_8)); } private void handleWrite(SelectionKey key) throws IOException { SocketChannel client = (SocketChannel) key.channel(); ConnectionState state = (ConnectionState) key.attachment(); if (state.writeBuffer == null) { return; } client.write(state.writeBuffer); if (!state.writeBuffer.hasRemaining()) { // Response sent, switch back to reading state.writeBuffer = null; state.readBuffer.clear(); key.interestOps(SelectionKey.OP_READ); } } private void closeConnection(SelectionKey key) { SocketChannel client = (SocketChannel) key.channel(); connections.remove(client); activeConnections.decrementAndGet(); key.cancel(); try { client.close(); } catch (IOException e) { // Already closing } } private static class ConnectionState { ByteBuffer readBuffer = ByteBuffer.allocate(BUFFER_SIZE); ByteBuffer writeBuffer = null; } public static void main(String[] args) throws IOException { new EventLoopHttpServer().start(); } } The event-loop server listens on port 8081. This command is a quick exploratory run; the recorded comparison later in the article comes from the repository\u0026rsquo;s multi-phase benchmark suite.\n1 bombardier -c 10000 -d 30s http://localhost:8081/ Each connection is represented by state such as READING → WRITING → READING. The event loop transitions that state when I/O is ready instead of assigning a platform thread to each connection.\nVirtual Threads vs Event Loops: The Real Trade-offs The comparison is easier to reason about as a set of trade-offs:\nAspect Virtual Threads (Blocking I/O) Event Loops (Non-blocking I/O) Programming Model Sequential, imperative Callback-based, state machines Memory model Stack chunks plus request state Explicit connection state and buffers Scheduling work Virtual-thread scheduling and mount/unmount behavior Event dispatch and state-machine transitions Debuggability Sequential control flow and familiar stack traces Traces can cross asynchronous boundaries Connection ceiling Depends on heap, workload, and runtime behavior Depends on state, buffers, OS limits, and workload Code complexity Often simpler for branching business logic Requires disciplined non-blocking handlers and state management Useful starting point Blocking request handlers and service orchestration Connection-heavy infrastructure with controlled I/O scheduling When to Use Virtual Threads I use virtual threads when:\nComplex business logic: Multiple database calls, service calls, and branching logic can be easier to follow in sequential code.\nFor example, a request that coordinates several remote calls can retain ordinary control flow while each virtual thread waits independently. Whether that improves maintainability depends on the framework, observability, and team conventions.\nBlocking I/O workloads: Virtual threads are a reasonable model to test when requests spend much of their time waiting on supported blocking operations. Heap use, pinning, and downstream limits still need measurement.\nTeam velocity: Most developers understand sequential code. Onboarding is faster. Code reviews are easier. Bugs are simpler to fix.\nHere\u0026rsquo;s a basic HTTP server implementation using virtual threads:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 package org.example; import com.sun.net.httpserver.HttpServer; import java.io.IOException; import java.io.OutputStream; import java.net.InetSocketAddress; import java.nio.charset.StandardCharsets; import java.util.concurrent.Executors; public class VirtualThreadsHttpServer { private static final int PORT = 8080; public static void main(String[] args) throws IOException { HttpServer server = HttpServer.create(new InetSocketAddress(PORT), 0); // Use virtual thread executor server.setExecutor(Executors.newVirtualThreadPerTaskExecutor()); server.createContext(\u0026#34;/\u0026#34;, exchange -\u0026gt; { // Simulate some work (database call, REST call, etc.) // This would block a platform thread, but virtual thread unmounts simulateWork(); String response = buildResponse(); exchange.getResponseHeaders().set(\u0026#34;Content-Type\u0026#34;, \u0026#34;application/json\u0026#34;); exchange.sendResponseHeaders(200, response.length()); try (OutputStream os = exchange.getResponseBody()) { os.write(response.getBytes(StandardCharsets.UTF_8)); } }); server.start(); System.out.println(\u0026#34;Virtual Threads HTTP Server started on port \u0026#34; + PORT); System.out.println(\u0026#34;Available processors: \u0026#34; + Runtime.getRuntime().availableProcessors()); // Print stats every 10 seconds Thread.startVirtualThread(() -\u0026gt; { while (true) { try { Thread.sleep(10_000); System.out.printf(\u0026#34;Stats - Active threads: %d, Virtual threads: yes%n\u0026#34;, Thread.activeCount()); } catch (InterruptedException e) { break; } } }); } private static void simulateWork() { // Simulate blocking I/O (database query, REST call, etc.) try { Thread.sleep(10); // 10ms simulated I/O } catch (InterruptedException e) { Thread.currentThread().interrupt(); } } private static String buildResponse() { return String.format( \u0026#34;{\\\u0026#34;message\\\u0026#34;:\\\u0026#34;Hello from Virtual Threads\\\u0026#34;,\\\u0026#34;thread\\\u0026#34;:\\\u0026#34;%s\\\u0026#34;,\\\u0026#34;timestamp\\\u0026#34;:%d}\u0026#34;, Thread.currentThread().toString(), System.currentTimeMillis() ); } } Total blocking time: ~10ms per request. With platform threads, this ties up a thread for 10ms. With virtual threads, the carrier thread stays free. The virtual thread unmounts at each blocking call. Other virtual threads run.\nWhen to Use Event Loops I use event loops when:\nConnection-heavy workloads: Event loops are worth testing when explicit control over I/O scheduling and per-connection state is more important than sequential request code.\nSimple request/response patterns: API gateways, load balancers, WebSocket servers, streaming proxies. The logic is simple: read request, forward it, write response. State machines work fine here.\nExplicit resource control: A small event-loop pool can make thread ownership predictable, but buffers, queues, callbacks, and application state still consume memory. Benchmark the complete process rather than comparing thread counts alone.\nThe Hybrid Approach A system can use both models. An event loop can own network I/O while other executors handle work that would otherwise block the loop. The handoff adds coordination cost, so it should be justified by measurements and framework behavior.\nPattern:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 // Event loop handles network I/O (Netty\u0026#39;s EventLoopGroup) ServerBootstrap bootstrap = new ServerBootstrap(); bootstrap.group(bossGroup, workerGroup) // Event loops .channel(NioServerSocketChannel.class) .childHandler(new ChannelInitializer\u0026lt;SocketChannel\u0026gt;() { @Override public void initChannel(SocketChannel ch) { ch.pipeline().addLast(new HttpServerCodec()); ch.pipeline().addLast(new HttpObjectAggregator(65536)); ch.pipeline().addLast(new SimpleChannelInboundHandler\u0026lt;FullHttpRequest\u0026gt;() { @Override protected void channelRead0(ChannelHandlerContext ctx, FullHttpRequest req) { // Dispatch business logic to virtual thread pool Thread.startVirtualThread(() -\u0026gt; { var response = handleBusinessLogic(req); ctx.writeAndFlush(response); }); } }); } }); Event loops handle I/O multiplexing. Virtual threads handle business logic. Best of both worlds.\nBenchmarks and Repo: Putting Both to the Test To validate the trade-offs with real numbers, I added a benchmark suite to a small project that runs both implementations side by side. The repo is virtual-thread-eventloop-test and is set up so you can run the same tests and draw your own conclusions.\nRepo Layout Here is the project link GITHUB .The project contains two HTTP servers and a 4-phase benchmark suite:\nVirtualThreadsHttpServer (port 8080) Java 21 HttpServer with Executors.newVirtualThreadPerTaskExecutor(). Each request runs on a virtual thread and does ~10 ms simulated blocking work (e.g. DB/REST). Simple sequential handler. EventLoopHttpServer (port 8081) Single-thread NIO server: one Selector, non-blocking ServerSocketChannel/SocketChannel, and the same 10 ms work simulated inside the event loop (no virtual threads). Pure reactor style. Both servers expose the same JSON endpoint and the same simulated workload so the comparison is about concurrency model, not API shape. Build with Maven; the pom.xml produces two runnable JARs: virtual-thread-app and event-loop-app.\nBenchmark Suite (4 Phases) The benchmarks folder holds a hypothesis-driven suite that measures throughput, latency, and resource use:\nPhase What it does Goal Phase 1: Baseline Fixed loads (e.g. 100, 1K, 10K connections), 10s–300s, multiple runs Establish normal throughput and latency patterns. Phase 2: Progressive stress Ramp connections from 100 → 50K (e.g. +1K every 30s) Find where each implementation degrades or fails. Phase 3: Spike Baseline (1K conn) → spike (10K conn) → back to 1K, repeated cycles Observe recovery and stability. Phase 4: Endurance Constant load (e.g. 5K connections) for several hours per server Check for memory growth and long-term stability. Load is generated with Bombardier (Go-based HTTP benchmark). The repo includes bombardier.exe in the scripts for Windows, so you can run the suite natively. The suite can collect JFR, JMX (e.g. VisualVM), and system metrics; the analyze-and-report script turns raw results into CSVs and a FINAL-REPORT.md in benchmark-results/.../analysis/.\nResults From a Sample Run Test machine (from the benchmark run’s config.json): 16 CPU cores, 31.82 GB RAM, 183 GB free disk. Each server ran with 4 GB heap (-Xmx4096m). Bombardier used 14 worker threads. Windows host.\nFrom one full run (Phase 1–3; Phase 2 summary and report):\nPeak throughput: Event Loop ~4,627 req/s vs Virtual Threads ~3,926 req/s event loop ahead under this workload. Breaking point (Phase 2): Both hit limits around 15,000 connections in that environment (stress ramp). Winner in this setup: Event Loop, for peak RPS, with both degrading at similar connection counts. So for this “many connections, small fixed delay per request” scenario, the single-thread event loop gave higher throughput, while virtual threads stayed in the same ballpark and remained predictable. Your mileage will depend on hardware, OS, and actual workload (e.g. real DB or HTTP calls).\nHow to Run It Yourself Clone/build: Open the virtual-thread-eventloop-test repo, build with Maven (mvn package). Use JDK 21+. Start both servers with enough heap (e.g. -Xmx4096m) and JMX if you want VisualVM. Virtual threads on 8080, event loop on 8081. Run the benchmark orchestrator from the benchmarks folder (see README.md and QUICK-REFERENCE.md). The suite uses Bombardier for load generation (e.g. bombardier.exe on Windows); ensure both servers are reachable from the machine running the benchmark. Analyze: Run the analysis script to generate FINAL-REPORT.md and the CSV summaries under benchmark-results/.../analysis/. The README in benchmarks explains the hypothesis template (predict VT vs EL before running), what to monitor in VisualVM, and how to interpret throughput, latency percentiles, and breaking points. Repeating the suite on your own machine is a good way to see how the two models behave under your constraints.\nChoosing a model Here\u0026rsquo;s my mental model :\nFor request handlers that spend most of their time waiting on blocking I/O, virtual threads are a reasonable first model to test. Event loops remain useful when connection density, memory, or control over I/O scheduling dominates the design. Measure the real workload before choosing.\nTest event loops when: The service owns many connections, handlers can remain non-blocking, and control over I/O scheduling or per-connection state is central to the design.\nTest a hybrid when: The network layer benefits from event loops but some application work is clearer or safer on a separate executor. Include the handoff and queueing behavior in the test.\nThe right tool depends on the workload. Virtual threads make blocking I/O practical at higher concurrency, while event loops retain value when explicit I/O scheduling and connection state are the dominant concerns.\nWhat I Learned Virtual threads and event loops solve different problems:\nVirtual threads: Make many blocking-I/O workloads easier to express with sequential code. They reduce reliance on large platform-thread pools but do not remove downstream capacity limits or the need to observe runtime behavior.\nEvent loops: Centralize readiness-driven I/O and make connection state explicit. They can suit infrastructure-style workloads when handlers remain non-blocking and the state-machine complexity is acceptable.\nUnderstanding both models gives you the full picture of Java\u0026rsquo;s I/O concurrency landscape. You can make informed decisions based on your actual constraints, not hype or cargo-culting.\nNext time you\u0026rsquo;re designing a system, ask: What\u0026rsquo;s the connection pattern? What\u0026rsquo;s the business logic complexity? What are the memory constraints? Then choose the right model.\nBoth are tools. The benchmark gives you a starting point, not a verdict.\nLimitations The sample run used one Windows machine and synthetic request patterns. It does not cover production network latency, database contention, long-lived WebSocket traffic, container limits, or failure recovery. Re-run the suite with the traffic shape and resource limits that matter to your system.\n","permalink":"https://mohibul.pages.dev/posts/posts/java/100daysofjava/day99/","summary":"A measured comparison of virtual threads and event loops, with a public benchmark suite and guidance for testing each model against a real workload.","title":"Virtual threads and event loops solve different problems"},{"content":"I added .parallel() to a file-processing stream and recorded 60% CPU utilization. Processing 100 million rows took 5.2 seconds in that run. A fixed-batch Spliterator reduced the measured time, which led me to inspect how the source divided work.\nVersion context. The unknown-size splitting defect tracked as JDK-8280915 was fixed in JDK 19. This article examines the runtime and workload recorded below. The measurements do not show that every current JDK or every parallel stream has the historical defect.\nTested JDK: The original run artifacts do not preserve the exact JDK build. The article references Java 21 documentation, but that does not prove which runtime produced the measurements. I therefore treat the numbers as a historical workload result, not a claim about current JDK behavior.\nWhat I didn\u0026rsquo;t get at first: the default Java Stream API has a load balancing problem. When you call .parallel() on an I/O-based source like Files.lines(), the spliterator that divides work among threads uses a growing batch size strategy: 1, 2, 4, 8, 16\u0026hellip; up to 16,384 elements per batch.\nThis creates massive imbalance. One thread gets a 16K batch and grinds away for 200ms. The other seven threads finish their tiny batches in 10ms and sit idle, waiting.\nI found this out the hard way on the One Billion Row Challenge, which is exactly what it sounds like: process 1 billion rows of weather data as fast as possible. The task is simple: read a text file where each line is station_name;temperature, compute min/mean/max per station, output sorted results.\nTop solutions finish in under 2 seconds using Unsafe, memory-mapped I/O, and custom parsers. I wanted to understand the fundamentals first: why does the Stream API struggle with this workload, and can we fix it without ditching the abstraction?\nIn the recorded run, a 50-line custom Spliterator cut processing time by 40% and pushed CPU utilization from 60% to 95%. It changed the batching policy without using Unsafe or native code.\nThe load balancing problem: what actually happens First version looked like this:\n1 2 3 4 5 6 7 8 9 try (Stream\u0026lt;String\u0026gt; lines = Files.lines(Paths.get(\u0026#34;measurements.txt\u0026#34;))) { Map\u0026lt;String, DoubleSummaryStatistics\u0026gt; result = lines .parallel() .map(line -\u0026gt; line.split(\u0026#34;;\u0026#34;)) .collect(Collectors.groupingBy( parts -\u0026gt; parts[0], Collectors.summarizingDouble(parts -\u0026gt; Double.parseDouble(parts[1])) )); } Eight cores. Should be fast. Wasn\u0026rsquo;t.\nI opened jconsole to watch CPU. This is what I saw:\nCore 0: 95% ████████████████████\rCore 1: 95% ████████████████████\rCore 2: 95% ████████████████████\rCore 3: 15% ███░░░░░░░░░░░░░░░░░\rCore 4: 15% ███░░░░░░░░░░░░░░░░░\rCore 5: 15% ███░░░░░░░░░░░░░░░░░\rCore 6: 15% ███░░░░░░░░░░░░░░░░░\rCore 7: 15% ███░░░░░░░░░░░░░░░░░ Three cores maxed out. Five cores mostly idle. Average: ~60%.\nThe problem is the Spliterator, the thing that divides work among threads. When you call Files.lines().parallel(), you get a spliterator that doesn\u0026rsquo;t know the file size upfront. It can\u0026rsquo;t split the file into eight equal chunks because it\u0026rsquo;s reading sequentially.\nSo it uses a different strategy: growing batches. First split creates a batch of 1 element. Next split: 2 elements. Then 4, 8, 16, 32, 64\u0026hellip; up to 16,384.\nThis is arithmetic progression with a cap. The JDK hardcodes these numbers in AbstractSpliterator.\nWhy? To minimize split overhead. Creating a new batch has cost: allocating arrays, coordinating threads, managing the work queue. If batches are too small, you spend more time splitting than processing.\nBut this optimization backfires for I/O workloads. Here\u0026rsquo;s what happens:\nTimeline of a parallel stream with growing batches:\nTime 0ms: Thread 1 gets batch of 1 element (processes in 0.05ms)\rTime 1ms: Thread 2 gets batch of 2 elements (processes in 0.1ms)\rTime 2ms: Thread 3 gets batch of 4 elements (processes in 0.2ms)\rTime 3ms: Thread 4 gets batch of 8 elements (processes in 0.4ms)\r...\rTime 50ms: Thread 1 gets batch of 16,384 elements (processes in 820ms)\rTime 51ms: Thread 2 gets batch of 16,384 elements (processes in 820ms)\rTime 52ms: Thread 3 gets batch of 16,384 elements (processes in 820ms)\rTime 870ms: Threads 4-8 are idle, waiting for threads 1-3 to finish The last few threads to grab work get massive batches. Everyone else finishes early and waits.\nThis is the load balancing problem. The spliterator optimizes for split overhead, not work distribution.\nThe fix: stop growing batches Marko Topolnik figured this out in 2014. Don\u0026rsquo;t let batch size grow. Pick a size and stick with it.\nInstead of 1, 2, 4, 8, 16\u0026hellip; just do 10,000, 10,000, 10,000, 10,000.\nEvery thread gets the same amount of work. No imbalance. No idle cores.\nThe implementation is 50 lines:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 public abstract class FixedBatchSpliteratorBase\u0026lt;T\u0026gt; implements Spliterator\u0026lt;T\u0026gt; { private final int batchSize; private final int characteristics; private long estimatedSize; public FixedBatchSpliteratorBase(int characteristics, int batchSize, long estimatedSize) { this.characteristics = characteristics | SUBSIZED; this.batchSize = batchSize; this.estimatedSize = estimatedSize; } @Override public Spliterator\u0026lt;T\u0026gt; trySplit() { final HoldingConsumer\u0026lt;T\u0026gt; holder = new HoldingConsumer\u0026lt;\u0026gt;(); if (!tryAdvance(holder)) { return null; } final Object[] batch = new Object[batchSize]; int j = 0; do { batch[j] = holder.value; } while (++j \u0026lt; batchSize \u0026amp;\u0026amp; tryAdvance(holder)); if (estimatedSize != Long.MAX_VALUE) { estimatedSize -= j; } return Spliterators.spliterator(batch, 0, j, characteristics()); } @Override public long estimateSize() { return estimatedSize; } @Override public int characteristics() { return characteristics; } static final class HoldingConsumer\u0026lt;T\u0026gt; implements Consumer\u0026lt;T\u0026gt; { T value; @Override public void accept(T value) { this.value = value; } } } The base class handles batching. You implement tryAdvance() to read one element:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 public class FileLineSpliterator extends FixedBatchSpliteratorBase\u0026lt;String\u0026gt; { private final BufferedReader reader; public FileLineSpliterator(BufferedReader reader, int batchSize, long estimatedSize) { super(ORDERED | NONNULL | IMMUTABLE, batchSize, estimatedSize); this.reader = reader; } @Override public boolean tryAdvance(Consumer\u0026lt;? super String\u0026gt; action) { try { String line = reader.readLine(); if (line == null) { return false; } action.accept(line); return true; } catch (IOException e) { throw new UncheckedIOException(e); } } public static Stream\u0026lt;String\u0026gt; stream(Path path, int batchSize) throws IOException { BufferedReader reader = Files.newBufferedReader(path); long estimatedSize = Files.size(path) / 50; return StreamSupport.stream( new FileLineSpliterator(reader, batchSize, estimatedSize), true ).onClose(() -\u0026gt; { try { reader.close(); } catch (IOException e) { throw new UncheckedIOException(e); } }); } } That\u0026rsquo;s it. tryAdvance() reads one line. The base class calls it batchSize times, fills an array, hands that array to a thread. Repeat until the file ends. The estimatedSize is a rough guess (file size divided by 50, assuming ~50 bytes per line on average) that the ForkJoinPool uses to decide how aggressively to split work.\nWhat this actually does When you call .parallel() on this stream, the ForkJoinPool calls trySplit() repeatedly to create work for threads.\nWith the default spliterator, thread 1 gets 1 element, thread 2 gets 2, thread 3 gets 4, and so on. Thread 8 ends up with 16,384 elements and processes for 820ms while the others sit there.\nWith the fixed-batch spliterator:\nThread 1: 10,000 elements (~500ms) Thread 2: 10,000 elements (~500ms) Thread 3: 10,000 elements (~500ms) \u0026hellip; Thread 8: 10,000 elements (~500ms) They all finish at roughly the same time.\nNo idle cores. No wasted CPU.\nUsing it for the One Billion Row Challenge The challenge: read 1 billion rows of station;temperature, compute min/mean/max per station, output sorted.\nHere\u0026rsquo;s the code:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 public class OneBillionRowChallenge { public static void main(String[] args) throws IOException { Path inputFile = Paths.get(\u0026#34;measurements.txt\u0026#34;); long start = System.nanoTime(); Map\u0026lt;String, StationStats\u0026gt; results; try (Stream\u0026lt;String\u0026gt; lines = FileLineSpliterator.stream(inputFile, 10_000)) { results = lines .map(OneBillionRowChallenge::parseLine) .collect(Collectors.groupingByConcurrent( Measurement::station, Collector.of( StationStats::new, StationStats::accept, StationStats::combine ) )); } // Sort and print results.entrySet().stream() .sorted(Map.Entry.comparingByKey()) .forEach(entry -\u0026gt; { StationStats stats = entry.getValue(); System.out.printf(\u0026#34;%s=%.1f/%.1f/%.1f%n\u0026#34;, entry.getKey(), stats.min, stats.sum / stats.count, stats.max ); }); long end = System.nanoTime(); System.err.printf(\u0026#34;%.2f seconds%n\u0026#34;, (end - start) / 1_000_000_000.0); } static Measurement parseLine(String line) { int idx = line.indexOf(\u0026#39;;\u0026#39;); String station = line.substring(0, idx); double temp = Double.parseDouble(line.substring(idx + 1)); return new Measurement(station, temp); } record Measurement(String station, double temp) {} static class StationStats { double min = Double.POSITIVE_INFINITY; double max = Double.NEGATIVE_INFINITY; double sum = 0; long count = 0; void accept(Measurement m) { min = Math.min(min, m.temp); max = Math.max(max, m.temp); sum += m.temp; count++; } StationStats combine(StationStats other) { min = Math.min(min, other.min); max = Math.max(max, other.max); sum += other.sum; count += other.count; return this; } } } Changed one line: Files.lines(path).parallel() became FileLineSpliterator.stream(path, 10_000).\nResult: 40% faster.\nPicking the right batch size I tried different batch sizes on 100 million rows:\nBatch Size Time CPU Util Memory Default JDK 5.2s 60% 1.2 GB 1,000 4.1s 75% 1.5 GB 10,000 3.1s 95% 1.8 GB 50,000 3.4s 85% 2.1 GB 100,000 4.0s 70% 2.5 GB 10,000 was the sweet spot. 40% faster than default. CPU utilization jumped from 60% to 95%.\nWhy not 1,000? Too much overhead. Each batch requires:\nAllocating an Object[] array Filling it with elements Creating a new Spliterator wrapper Submitting it to the ForkJoinPool At 1,000 elements per batch, you\u0026rsquo;re creating 100,000 batches for 100M rows. That\u0026rsquo;s 100,000 array allocations, 100,000 spliterator objects, 100,000 pool submissions. The GC can\u0026rsquo;t keep up. You spend more time managing batches than processing data.\nWhy not 100,000? Back to the load imbalance problem. If each batch takes 5 seconds to process and you have 8 threads, the last thread to get work might process for 5 seconds while others sit idle.\nTopolnik\u0026rsquo;s rule: each batch should take 1–10 ms to process. That balances batch overhead (allocation, coordination, GC) against load imbalance (idle threads waiting).\nFor this workload on my machine, 10,000 rows takes about 5ms to process. Perfect.\nYour optimal size depends on:\nHow expensive each row is to process Your core count (more cores = need more batches) Your I/O speed (faster I/O = can handle smaller batches) Measure. Don\u0026rsquo;t guess.\nWhere the time goes I ran this through JFR (Java Flight Recorder) to see the breakdown:\nTotal: 3.1 seconds for 100M rows\rI/O (BufferedReader.readLine): 0.8s (26%)\rParsing (indexOf + substring + parse): 1.4s (45%)\rAggregation (ConcurrentHashMap): 0.6s (19%)\rThread coordination: 0.2s (6%)\rGC: 0.1s (3%) Parsing is the bottleneck. 45% of time goes to indexOf(), substring(), and Double.parseDouble().\nThis surprised me. I thought I/O would dominate. Nope. Modern SSDs are fast; sequential reads hit 3–5 GB/s. Reading 5 GB of text takes under 2 seconds.\nBut parsing? Every row requires:\nindexOf(';') - scan bytes looking for semicolon substring(0, idx) - allocate new String for station name substring(idx + 1) - allocate new String for temperature Double.parseDouble() - parse string to double That\u0026rsquo;s three string allocations per row. 100 million rows = 300 million strings. Each string is 16 bytes of object header + char array + length. That\u0026rsquo;s ~5 GB of allocations just for parsing.\nThe GC handles it (only 3% overhead) because these strings are short-lived. But the allocation and copying still burns CPU.\nTop 1BRC solutions don\u0026rsquo;t use String.split() or substring(). They parse directly from byte arrays. No allocations. Just pointer arithmetic and manual digit parsing.\nThat\u0026rsquo;s the next optimization. But it requires abandoning the Stream API and working with raw byte buffers.\nHow top solutions get under 2 seconds My fixed-batch implementation: 3.1 seconds for 100M rows. Extrapolate to 1B rows: ~31 seconds.\nTop 1BRC solutions: under 2 seconds for 1B rows.\nWhat are they doing?\n1. Custom Parsing They don\u0026rsquo;t use String.split() or Double.parseDouble(). They parse bytes directly. Here\u0026rsquo;s what I do:\n1 2 3 int idx = line.indexOf(\u0026#39;;\u0026#39;); String station = line.substring(0, idx); double temp = Double.parseDouble(line.substring(idx + 1)); Here\u0026rsquo;s what top solutions do:\n1 2 3 int idx = findSemicolon(bytes, offset); long stationHash = hashBytes(bytes, offset, idx); double temp = parseDouble(bytes, idx + 1); No string allocations. No substring copies. Just pointer arithmetic and manual digit parsing.\nThey also use hash codes instead of strings for station names. Store the hash in a primitive long array. Only materialize the actual string at the end for output.\nRoughly 30–40% of parsing time saved.\n2. Memory-Mapped I/O Instead of BufferedReader, they use FileChannel.map():\n1 2 FileChannel channel = FileChannel.open(path, StandardOpenOption.READ); MappedByteBuffer buffer = channel.map(FileChannel.MapMode.READ_ONLY, 0, channel.size()); The OS maps the file directly into process memory. No read() syscalls. No copying from kernel space to user space. Just pointer dereferences.\nAbout 10–15% of I/O time saved.\n3. Thread-Local Aggregation ConcurrentHashMap has overhead. Every put() or compute() requires:\nHash calculation CAS operation to check for contention Potential lock acquisition if bucket is contested Instead, each thread maintains its own HashMap. No contention. No CAS. No locks. At the end, merge all maps single-threaded.\n1 2 3 4 5 6 7 8 Map\u0026lt;String, StationStats\u0026gt;[] localMaps = new Map[numThreads]; Map\u0026lt;String, StationStats\u0026gt; result = new HashMap\u0026lt;\u0026gt;(); for (Map\u0026lt;String, StationStats\u0026gt; local : localMaps) { local.forEach((station, stats) -\u0026gt; result.merge(station, stats, StationStats::combine) ); } About half of aggregation time saved.\n4. Unsafe Top solutions use sun.misc.Unsafe to:\nRead memory without bounds checks Parse numbers with manual bit manipulation Use SIMD instructions for scanning bytes This is not portable. Not safe. But fast.\nAnother 20–30% on top of that.\n5. GraalVM Native Image Compile to native binary. No JVM startup. No JIT warmup. No interpreter.\n1–2 seconds of fixed overhead gone.\nThe performance ladder Here\u0026rsquo;s the full spectrum for 1 billion rows:\nApproach Time Complexity Portable? Sequential 180s Low Yes Default .parallel() 52s Low Yes Fixed-batch spliterator 31s Medium Yes + Custom parsing 22s High Yes + Memory-mapped I/O 19s High Yes + Thread-local aggregation 13s High Yes + Unsafe 5s Very High No + GraalVM native 2s Very High No Fixed-batch is the sweet spot. 3.5x faster than default. Still maintainable. Still portable.\nEverything after that trades maintainability for speed. Custom parsing means you\u0026rsquo;re writing your own indexOf() and parseDouble(). Memory-mapped I/O means you\u0026rsquo;re managing byte buffers and offsets. Thread-local aggregation means you\u0026rsquo;re manually partitioning work.\nAnd Unsafe? You\u0026rsquo;re reading raw memory addresses. One off-by-one error and you segfault the JVM.\nI\u0026rsquo;m comfortable stopping at fixed-batch for production code. The rest is for competitions and benchmarks.\nWhen this actually helps Fixed-batch spliterators make sense when you\u0026rsquo;re reading from I/O (files, network streams, database cursors) and can\u0026rsquo;t split the source upfront. They also help when per-element work is non-trivial (parsing, validation, transformation). If one element takes under a microsecond, batch overhead dominates. You need real parallelism: multiple cores and a workload that\u0026rsquo;s CPU-bound after I/O. And you want something maintainable: standard Java, no Unsafe, nothing that makes code review painful.\nSkip it when your data is already in memory (arrays, lists, collections split fine on their own). Skip it when processing is trivial (simple filters or no-op maps); batch overhead costs more than you gain. If you\u0026rsquo;re I/O bound, parallelism won\u0026rsquo;t help; profile first. And if you need predictable latency, batching adds variance: some requests hit batch boundaries, others don\u0026rsquo;t.\nThe memory cost Fixed-batch uses more memory than default streams.\nWhy? Each batch materializes into an Object[] array. With 8 threads and 10K batch size, you have 8 × 10K = 80K elements in memory.\nMeasured on the 1BRC:\nDefault stream: 1.2 GB peak Fixed-batch (10K): 1.8 GB peak Fixed-batch (50K): 2.5 GB peak You\u0026rsquo;re trading memory for CPU utilization. Usually worth it—memory is cheap, CPU is expensive.\nBut if you\u0026rsquo;re memory-constrained (containers with tight limits, embedded systems), this matters. You might need smaller batches or a different approach.\nAlso: larger batches mean larger arrays. Large arrays can cause GC issues:\nYoung gen collections happen more often Arrays might get promoted to old gen Fragmentation increases I saw GC overhead jump from 3% to 8% when I went from 10K to 100K batch size. The arrays were large enough to survive young gen collections and get promoted. Then old gen filled up and triggered full GCs.\nKeep batches small enough to stay in young gen. For most workloads, that\u0026rsquo;s under 100K elements.\nWhat surprised me For this workload, growing batches reduced split overhead but left work distributed unevenly. The fixed-batch implementation traded more splitting for steadier work distribution. That is a workload-specific trade-off, not a general verdict on parallel streams.\nI expected to spend most of my time waiting on disk. Nope. Parsing is the bottleneck. Modern SSDs do 3–5 GB/s sequential. Reading the file is a couple of seconds. Parsing is 14 seconds. String allocation, substring copies, parseDouble(). That\u0026rsquo;s where the time goes. Top solutions avoid strings and parse bytes directly. No allocations, just arithmetic.\nParallelism has steep diminishing returns. 1 thread to 8 threads: 8x (linear). Default parallel to fixed-batch: 1.7x. Fixed-batch to custom parsing: 1.4x. Custom parsing to Unsafe: 2.6x, but 10x the complexity. Each step is harder and gives less. Fixed-batch is the last easy win.\nThe 1BRC isn\u0026rsquo;t about algorithms. It\u0026rsquo;s about constants. The gap between 30 seconds and 2 seconds is 3 string allocations per row vs zero, ConcurrentHashMap CAS vs thread-local HashMap, bounds-checked access vs Unsafe, JIT vs GraalVM native. Every microsecond counts.\nWhat I\u0026rsquo;d do differently in production For a system that processes large files (logs, CSV exports, data dumps), I\u0026rsquo;d start with the fixed-batch spliterator. Fifty lines, 40% speedup, still maintainable, runs on any JVM. If that\u0026rsquo;s not enough, custom parsing—bytes directly, no string allocations—but it\u0026rsquo;s easy to get wrong, so test carefully. If it\u0026rsquo;s still not enough, step back: do you really need to process 1B rows in one shot? Can you stream, preprocess, or partition across machines?\nUnsafe and memory-mapped I/O are last resorts. The complexity and maintenance cost are real. I\u0026rsquo;ve seen teams spend weeks tuning hot paths with Unsafe only to find the real bottleneck somewhere else. Profile first. Fix the algorithm and data structures. Then micro-optimize.\nThe 1BRC rewards extreme optimization. Production rewards maintainability and correctness. Fixed-batch sits in the middle.\nThe complete implementation Here\u0026rsquo;s everything you need to benchmark this yourself. The benchmark helper runs 3 warmup iterations to let the JIT compile the hot path, then 5 timed runs and reports the median:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 public class BenchmarkFixedBatch { public static void main(String[] args) throws IOException { int numRows = args.length \u0026gt; 0 ? Integer.parseInt(args[0]) : 1_000_000_000; int[] batchSizes = args.length \u0026gt; 1 ? parseBatchSizes(args[1]) : new int[]{1_000, 10_000, 50_000, 100_000}; Path testFile = generateTestFile(numRows); System.out.println(\u0026#34;Benchmarking \u0026#34; + String.format(\u0026#34;%,d\u0026#34;, numRows) + \u0026#34; rows...\\n\u0026#34;); benchmark(\u0026#34;Sequential\u0026#34;, () -\u0026gt; processSequential(testFile)); benchmark(\u0026#34;Default Parallel\u0026#34;, () -\u0026gt; processDefaultParallel(testFile)); for (int batchSize : batchSizes) { benchmark(\u0026#34;Fixed-Batch \u0026#34; + batchSize, () -\u0026gt; processFixedBatch(testFile, batchSize)); } } private static int[] parseBatchSizes(String arg) { String[] parts = arg.split(\u0026#34;,\u0026#34;); int[] sizes = new int[parts.length]; for (int i = 0; i \u0026lt; parts.length; i++) { sizes[i] = Integer.parseInt(parts[i].trim()); } return sizes; } private static void benchmark(String name, ThrowingRunnable task) { for (int i = 0; i \u0026lt; 3; i++) { try { task.run(); } catch (Exception e) { throw new RuntimeException(e); } } long[] times = new long[5]; for (int i = 0; i \u0026lt; 5; i++) { long start = System.nanoTime(); try { task.run(); } catch (Exception e) { throw new RuntimeException(e); } times[i] = System.nanoTime() - start; } Arrays.sort(times); double median = times[2] / 1_000_000_000.0; System.out.printf(\u0026#34;%-20s: %.2f seconds%n\u0026#34;, name, median); } private static Map\u0026lt;String, StationStats\u0026gt; processSequential(Path path) throws IOException { try (Stream\u0026lt;String\u0026gt; lines = Files.lines(path)) { return lines .map(OneBillionRowChallenge::parseLine) .collect(Collectors.groupingBy( Measurement::station, Collector.of( StationStats::new, StationStats::accept, StationStats::combine ) )); } } private static Map\u0026lt;String, StationStats\u0026gt; processDefaultParallel(Path path) throws IOException { try (Stream\u0026lt;String\u0026gt; lines = Files.lines(path)) { return lines .parallel() .map(OneBillionRowChallenge::parseLine) .collect(Collectors.groupingByConcurrent( Measurement::station, Collector.of( StationStats::new, StationStats::accept, StationStats::combine ) )); } } private static Map\u0026lt;String, StationStats\u0026gt; processFixedBatch(Path path, int batchSize) throws IOException { try (Stream\u0026lt;String\u0026gt; lines = FileLineSpliterator.stream(path, batchSize)) { return lines .map(OneBillionRowChallenge::parseLine) .collect(Collectors.groupingByConcurrent( Measurement::station, Collector.of( StationStats::new, StationStats::accept, StationStats::combine ) )); } } private static Path generateTestFile(int numRows) throws IOException { Path path = Paths.get(\u0026#34;test_measurements_\u0026#34; + numRows + \u0026#34;.txt\u0026#34;); if (Files.exists(path)) return path; String[] stations = { \u0026#34;Hamburg\u0026#34;, \u0026#34;Bulawayo\u0026#34;, \u0026#34;Palembang\u0026#34;, \u0026#34;St. John\u0026#39;s\u0026#34;, \u0026#34;Cracow\u0026#34;, \u0026#34;Bridgetown\u0026#34;, \u0026#34;Istanbul\u0026#34;, \u0026#34;Roseau\u0026#34;, \u0026#34;Conakry\u0026#34;, \u0026#34;Ankara\u0026#34; }; Random random = new Random(42); try (BufferedWriter writer = Files.newBufferedWriter(path)) { for (int i = 0; i \u0026lt; numRows; i++) { String station = stations[random.nextInt(stations.length)]; double temp = -10 + random.nextDouble() * 50; writer.write(String.format(\u0026#34;%s;%.1f%n\u0026#34;, station, temp)); } } return path; } @FunctionalInterface interface ThrowingRunnable { void run() throws Exception; } } Run this. Watch your CPU utilization jump from 60% to 95%.\nDay 100: what I learned about performance Optimization is a ladder: right algorithm, right data structures, right concurrency, then constants (allocation, parsing, copying), then the metal (Unsafe, SIMD, native). Most code never gets past step 2. Some needs step 3. Very little needs 4 or 5.\nFixed-batch spliterators sit around 3.5. Fifty lines, 40% faster. The 1BRC made it clear there\u0026rsquo;s always another level—custom parsing, memory-mapped I/O, thread-local aggregation, Unsafe, GraalVM. Each step is harder and pays off less. Knowing when to stop is the part nobody teaches.\nLimitations The original run does not preserve the exact JDK build, raw benchmark files, or enough repeated trials to estimate variance. The percentages describe one recorded machine and workload. They do not establish performance on JDK 19 or later, other file sizes, different storage, or non-file stream sources.\nOpenJDK fixed the related unknown-size splitting defect in JDK-8280915 in JDK 19. Re-run the benchmark on the current target JDK before choosing a custom Spliterator.\nReferences Details and inspiration from:\nOne Billion Row Challenge Repository - Gunnar Morling\u0026rsquo;s challenge and leaderboard Parallel Processing of I/O-Based Data with Java Streams - Marko Topolnik\u0026rsquo;s original fixed-batch spliterator article Java Spliterator Documentation - Official Java documentation JEP 266: More Concurrency Updates - Background on Java concurrency improvements 1BRC Solutions Analysis - Detailed breakdown of optimization techniques Java Performance Tuning Guide - Oracle\u0026rsquo;s performance best practices ","permalink":"https://mohibul.pages.dev/posts/posts/java/100daysofjava/day100-spliterator/","summary":"Why one parallel file-processing run left CPU cores idle, how a fixed-batch Spliterator changed the result, and why the finding is tied to its runtime and workload.","title":"Why parallel file processing left CPU cores idle"},{"content":"I Thought Virtual Threads Were Just Green Threads—They\u0026rsquo;re Actually Continuations on the Heap I used virtual threads in an API earlier in this challenge. Spun up a few thousand threads, watched memory stay flat, called it a win. My mental model was simple: virtual threads are like Go\u0026rsquo;s goroutines—lightweight threads the runtime manages instead of the OS. Green threads. Done.\nFor thos who want to learn more about goroutines you can look into these videos to learn more about concurrency in go http://youtube.com/watch?v=5zXAHh5tJqQ\nThen I tried to understand why a synchronized block could pin a virtual thread in Java 21 but not in Java 24.\nBefore that lets know what is pinnig a thread means. Pinning, the binding of a process or thread to a specific core, can improve the performance of your code by increasing the percentage of local memory accesses. But in java it is preventing a virtual thread from being unmounted from its carrier thread\nThat question pulled me into the underlying implementation, and I realized my mental model was completely wrong. Virtual threads aren\u0026rsquo;t just \u0026ldquo;lighter\u0026rdquo; OS threads. They\u0026rsquo;re heap-allocated continuation objects with their own stack chunks, scheduled by a work-stealing ForkJoinPool, with mount/unmount semantics\nHere\u0026rsquo;s what I got wrong and what actually happens under the hood.\nThe Mental Model I Had (Wrong) Before this deep dive, here\u0026rsquo;s what I thought:\nVirtual threads = JVM-managed lightweight threads (like goroutines) Main benefit = no 1MB stack reservation per thread They \u0026ldquo;just work\u0026rdquo; with existing blocking code Implementation details = \u0026ldquo;JVM magic\u0026rdquo; I didn\u0026rsquo;t need to understand This model worked for basic usage. I could write Thread.ofVirtual().start(() -\u0026gt; ...) and handle thousands of concurrent requests without thinking about thread pools. Good enough.\nBut it didn\u0026rsquo;t explain the weird constraints I kept seeing:\nWhy does synchronized pin threads in Java 21? Why do people say \u0026ldquo;use ReentrantLock instead\u0026rdquo;? Why does the documentation warn about ThreadLocal with millions of threads? Why are stacks on the heap instead of\u0026hellip; wherever stacks normally live? The \u0026ldquo;green threads\u0026rdquo; model had no answers. Time to dig deeper.\nWhat Virtual Threads Actually Are: Continuations The core primitive isn\u0026rsquo;t a thread at all—it\u0026rsquo;s a continuation. This is the part that broke my mental model.\nA continuation is a Java object that captures execution state: local variables, method call frames, the instruction pointer. When a virtual thread blocks on I/O, the JVM doesn\u0026rsquo;t let it block the underlying OS thread. Instead, the continuation object captures the current state, the virtual thread \u0026ldquo;unmounts\u0026rdquo; from its carrier thread, and the carrier becomes available for other virtual threads.\nHere\u0026rsquo;s the internal API (not accessible to user code, but this is what happens):\n1 2 3 4 5 6 7 // What the JVM does internally when you create a virtual thread var scope = new ContinuationScope(\u0026#34;VirtualThreads\u0026#34;); var continuation = new Continuation(scope, () -\u0026gt; { // Your code runs here socket.read(); // Blocking operation triggers yield // Execution resumes here later, possibly on different carrier }); When socket.read() blocks, the continuation calls yield(scope). This doesn\u0026rsquo;t block the OS thread—it saves the execution state to the heap and returns control to the scheduler. Later, when the I/O completes, the scheduler picks an available carrier thread and resumes the continuation from where it left off.\nThis is not how platform threads work. Platform threads block the OS thread. The OS kernel handles the context switch. With virtual threads, the JVM handles everything in user space.\nThe Stack Problem: Why Heap Allocation Matters Here\u0026rsquo;s the memory efficiency piece I thought I understood but didn\u0026rsquo;t.\nPlatform threads allocate their stacks in native OS memory—typically 1MB reserved at creation time, whether you use it or not. If you create 10,000 platform threads, you\u0026rsquo;ve reserved 10GB of native memory for stacks alone, even if most threads are blocked waiting for I/O with shallow call stacks.\nVirtual threads store their stacks as stack chunk objects on the garbage-collected heap. These chunks grow dynamically based on actual call depth:\nShallow I/O operation: ~1KB heap allocation Deep call chain: ~50KB heap allocation Deeply recursive algorithm: up to ~512KB (G1 GC region size limit) The math changes dramatically:\n10,000 platform threads: ~10GB native memory (reserved, not used) 10,000 virtual threads: ~10MB heap memory (actual usage) But here\u0026rsquo;s the part I missed: this creates work during mount/unmount operations. When a virtual thread mounts on a carrier, the JVM copies stack frames from the heap to the carrier\u0026rsquo;s native stack. When it unmounts, frames copy back to the heap. This is fast (1-5 microseconds) but not free.\nThe trade-off: memory efficiency for CPU overhead during context switches. For I/O-bound workloads where threads spend most of their time blocked, this trade-off is massively in your favor. For CPU-bound workloads, you\u0026rsquo;re just burning cycles copying stacks around.\nThe Scheduler: ForkJoinPool in FIFO Mode I assumed virtual threads had some custom scheduler. Nope—it\u0026rsquo;s a ForkJoinPool, the same work-stealing pool used by parallel streams, but configured differently.\nKey differences from the common pool:\nFIFO mode instead of LIFO (better for long-running tasks) Default parallelism = number of CPU cores Dynamically expands when threads pin (more on this shortly) The work-stealing algorithm means idle carrier threads steal virtual threads from busy carriers\u0026rsquo; queues. This keeps all carriers busy and improves utilization.\nBut here\u0026rsquo;s the critical constraint: virtual threads are cooperative, not preemptive. If a virtual thread runs CPU-bound code without calling any blocking operations, it monopolizes its carrier. No time-slicing. No forced context switch. This can starve other virtual threads.\nI\u0026rsquo;ve always had preemptive thread scheduling. Virtual threads don\u0026rsquo;t work that way. If you have a CPU-bound task, use a platform thread or parallel stream. Virtual threads are for I/O-bound concurrency.\nThe Pinning Problem (And Why Java 24 Matters) This is where my \u0026ldquo;green threads\u0026rdquo; model completely fell apart.\nIn Java 21-23, if a virtual thread blocks inside a synchronized block, it pins to its carrier thread. The virtual thread cannot unmount. The carrier thread blocks. You\u0026rsquo;ve just turned your virtual thread back into a platform thread, defeating the entire purpose.\nWhy? Because synchronized monitors are tied to the OS thread identity. The JVM couldn\u0026rsquo;t separate the monitor state from the carrier thread. If the virtual thread unmounted, the monitor would be released incorrectly.\nThis is a massive problem if you\u0026rsquo;re working with legacy code full of synchronized blocks. Your virtual threads pin constantly, the scheduler compensates by expanding the carrier pool beyond the CPU count, and you end up with hundreds of OS threads anyway.\nThe workaround in Java 21: replace synchronized with ReentrantLock, which properly unmounts virtual threads. But that\u0026rsquo;s a non-trivial refactor if you have thousands of synchronized methods.\nJava 24 (JEP 491) fixes this. Virtual threads can now hold monitors independently of their carriers. When a virtual thread blocks inside synchronized code, it unmounts normally. The monitor state is tracked by the JVM, not the carrier thread. When the lock is acquired, the virtual thread resumes on any available carrier.\nThis was the implementation detail that made me realize how sophisticated this system is. The JVM had to fundamentally change how monitors work to make virtual threads scale with real-world Java code.\nWhat I Measured (And What Surprised Me) I wanted to see the memory difference myself, so I wrote a simple test:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 // Platform threads ExecutorService platformPool = Executors.newFixedThreadPool(200); for (int i = 0; i \u0026lt; 10_000; i++) { platformPool.submit(() -\u0026gt; { try { Thread.sleep(10_000); } catch (InterruptedException e) {} }); } // Result: ~200MB native memory (200 threads × 1MB stacks) // 9,800 tasks queued, waiting for threads // Virtual threads ExecutorService virtualPool = Executors.newVirtualThreadPerTaskExecutor(); for (int i = 0; i \u0026lt; 10_000; i++) { virtualPool.submit(() -\u0026gt; { try { Thread.sleep(10_000); } catch (InterruptedException e) {} }); } // Result: ~10MB heap memory // All 10,000 threads sleeping concurrently The platform thread version queues 9,800 tasks because only 200 threads exist. The virtual thread version runs all 10,000 concurrently because creating threads is cheap.\nWhat surprised me: CPU utilization with virtual threads was higher (85% vs 60%) on an I/O-bound workload. The platform thread pool left cores idle waiting for threads to unblock. Virtual threads kept carriers busy by unmounting blocked threads and scheduling runnable ones.\nThe Trade-offs I Didn\u0026rsquo;t Expect Virtual threads are not always better. Here\u0026rsquo;s what I learned:\nUse virtual threads when:\nI/O-bound workloads (network calls, database queries, file I/O) You need millions of concurrent operations Blocking code is simpler than async pipelines You\u0026rsquo;re on Java 24+ (or can avoid synchronized) Use platform threads when:\nCPU-bound workloads (no blocking = no benefit from unmounting) You need thread priorities or custom scheduling You\u0026rsquo;re calling native code frequently (FFM/JNI pins threads) You have deeply recursive algorithms (stack size limits) The ThreadLocal trap: If you use ThreadLocal variables with millions of virtual threads, you\u0026rsquo;re allocating millions of ThreadLocal entries on the heap. This can create memory pressure. Java 21 introduced Scoped Values as a better alternative for virtual thread scenarios—lexically scoped, no per-thread allocation.\nThe debugging challenge: Thread dumps with millions of virtual threads are\u0026hellip; interesting. The new JSON format helps, but tooling is still catching up. Structured concurrency (preview feature) helps by establishing task hierarchies debuggers can visualize.\nWhat I\u0026rsquo;ll Do Differently Before this deep dive, I treated virtual threads as a drop-in replacement for thread pools. Now I know better.\nFor new I/O-bound services: Virtual threads by default. The simplicity of thread-per-request code with the scalability of async systems is worth it.\nFor existing services: Check for synchronized blocks first. If we\u0026rsquo;re on Java 24+, virtual threads are safe. If we\u0026rsquo;re stuck on Java 21-23, I need to profile for pinning (-Djdk.tracePinnedThreads=full) and decide if the refactor to ReentrantLock is worth it.\nFor CPU-bound tasks: Stick with platform threads or parallel streams. Virtual threads add overhead without benefit.\nFor mixed workloads: Separate pools. Virtual threads for I/O, platform threads for CPU work. Don\u0026rsquo;t mix them in the same executor.\nThe mental model shift matters. Virtual threads aren\u0026rsquo;t just \u0026ldquo;lighter threads\u0026rdquo;—they\u0026rsquo;re a different concurrency model with different trade-offs. Understanding continuations, heap-based stacks, and the mount/unmount mechanism helps me make better architectural decisions.\nNext time I see a service hitting thread pool limits, I\u0026rsquo;ll know exactly why virtual threads might help—and when they won\u0026rsquo;t.\nReferences Technical details verified against:\nJEP 444: Virtual Threads JEP 491: Synchronize Virtual Threads Without Pinning Foojay: Continuations Foundation of Virtual Threads Oracle Java Magazine: Virtual Threads Design nipafx: Virtual Thread Deep Dive InfoQ: Virtual Threads Lightweight Concurrency Cashfree: Production Lessons from Virtual Threads YCrash: Virtual Threads vs Platform Threads Study Stackademic: Spring Boot Virtual Threads Benchmarking Academic Study: Virtual Threads Performance Analysis ","permalink":"https://mohibul.pages.dev/posts/posts/java/100daysofjava/day98/","summary":"I thought virtual threads were just green threads—turns out they\u0026rsquo;re continuations on the heap with sophisticated scheduling. Here\u0026rsquo;s what I got wrong and what actually happens under the hood.","title":"Day 98: I Thought Virtual Threads Were Just Green Threads They're Actually Continuations on the Heap"},{"content":" Series Navigation: Part 1: Foundation \u0026amp; Execution • Part 2: Core Synchronization • Part 3 (You are here)\n\u0026ldquo;Mastering concurrency means knowing when NOT to use these tools, and how to debug them when things break.\u0026rdquo;\nIn Part 1 (Day 95), we explored execution patterns. In Part 2 (Day 96), we covered core synchronization tools.\nNow in Part 3, we tackle advanced patterns and the critical skills for production systems: debugging, monitoring, and operational readiness.\nSeries Overview This is a 3-part series on Java Concurrency:\nPart 1 (Day 95): Foundation \u0026amp; Execution\nExecutor, ExecutorService, ScheduledExecutorService Future, CompletableFuture, CountDownLatch Part 2 (Day 96): Core Synchronization Patterns\nCyclicBarrier, Semaphore, ThreadFactory, BlockingQueue Part 3 (Today - Day 97): Advanced Patterns \u0026amp; Production Readiness\nDelayQueue, ReentrantLock, Phaser Debugging, Monitoring, Production Patterns What We\u0026rsquo;re Covering Today Part 3 (Today): Advanced Patterns \u0026amp; Production Readiness\nDelayQueue - Time-delayed execution with exponential backoff ReentrantLock - Fine-grained locking control Phaser - Dynamic multi-phase coordination Debugging Concurrent Code Monitoring Concurrent Systems Production Readiness Checklist Quick Reference: Part 3 Tools at a Glance Tool TL;DR Key Gotcha DelayQueue Queue where items become available after a delay. Built for exponential backoff retries. Unbounded (grows indefinitely). Requires Delayed interface ReentrantLock Explicit locking with timeouts, interruptibility, fair mode. More power than synchronized. Must unlock in finally - no automatic unlock on exception Phaser Most advanced - dynamic party registration/deregistration, multiple phases. Most complex synchronizer. Use only when CountDownLatch/CyclicBarrier won\u0026rsquo;t work 1. DelayQueue: Exponential Backoff Retry The Problem: HTTP requests fail intermittently. Retry immediately and you overwhelm the failing service. Wait too long and users timeout.\nDelayQueue holds tasks until a delay expires. Tasks become available for processing only after their delay passes. No manual sleep() calls needed.\nHow it works: Each task stores its trigger time. take() call blocks until a task\u0026rsquo;s delay expires.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 import java.util.concurrent.*; public class RetryQueue { static class RetryTask implements Delayed { private final String payload; private final long triggerTime; private final int attemptNumber; public RetryTask(String payload, long delayMs, int attemptNumber) { this.payload = payload; this.triggerTime = System.currentTimeMillis() + delayMs; this.attemptNumber = attemptNumber; } @Override public long getDelay(TimeUnit unit) { long diff = triggerTime - System.currentTimeMillis(); return unit.convert(diff, TimeUnit.MILLISECONDS); } @Override public int compareTo(Delayed other) { return Long.compare(this.triggerTime, ((RetryTask) other).triggerTime); } public String getPayload() { return payload; } public int getAttemptNumber() { return attemptNumber; } } private final DelayQueue\u0026lt;RetryTask\u0026gt; retryQueue; private final ExecutorService worker; private final int maxRetries; public RetryQueue(int maxRetries) { this.retryQueue = new DelayQueue\u0026lt;\u0026gt;(); this.worker = Executors.newSingleThreadExecutor( new NamedThreadFactory(\u0026#34;retry-worker\u0026#34;, false, Thread.NORM_PRIORITY) ); this.maxRetries = maxRetries; } public void start() { worker.submit(() -\u0026gt; { while (!Thread.currentThread().isInterrupted()) { try { // Blocks until a task\u0026#39;s delay expires RetryTask task = retryQueue.take(); System.out.println(\u0026#34;Attempting delivery (attempt \u0026#34; + task.getAttemptNumber() + \u0026#34;): \u0026#34; + task.getPayload()); boolean success = sendWebhook(task.getPayload()); if (!success \u0026amp;\u0026amp; task.getAttemptNumber() \u0026lt; maxRetries) { // Exponential backoff: 2^attempt seconds long backoffMs = (long) Math.pow(2, task.getAttemptNumber()) * 1000; System.out.println(\u0026#34;Delivery failed. Retrying in \u0026#34; + backoffMs + \u0026#34;ms\u0026#34;); retryQueue.put(new RetryTask( task.getPayload(), backoffMs, task.getAttemptNumber() + 1 )); } else if (success) { System.out.println(\u0026#34;Delivery successful!\u0026#34;); } else { System.err.println(\u0026#34;Max retries exceeded. Giving up.\u0026#34;); } } catch (InterruptedException e) { Thread.currentThread().interrupt(); break; } } }); } public void scheduleDelivery(String payload) { retryQueue.put(new RetryTask(payload, 0, 1)); } private boolean sendWebhook(String payload) { // Simulate random failures return ThreadLocalRandom.current().nextDouble() \u0026gt; 0.3; } public void shutdown() { worker.shutdown(); } } Production note: This queue lives in memory. Application restart loses pending retries. Persist to Redis or a database if retries must survive restarts.\nTrade-offs What you gain:\nAutomatic delay enforcement. No manual sleep() or timers. Tasks ordered by trigger time. Earliest task dequeues first. Thread-safe. Multiple consumers can call take() concurrently. What you pay:\nUnbounded. No backpressure. Queue grows until memory exhausted. Clock-dependent. System time changes affect delays. CPU overhead. Internal heap reorders on every insert. Boilerplate. Must implement Delayed interface for every task type. Critical implementation detail:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 // WRONG: Stores relative delay class BadDelayedTask implements Delayed { private long delay; // 5000ms public long getDelay(TimeUnit unit) { return unit.convert(delay, TimeUnit.MILLISECONDS); // Always returns 5000ms. Task never becomes available. } } // CORRECT: Stores absolute trigger time class GoodDelayedTask implements Delayed { private final long triggerTime; // System.currentTimeMillis() + 5000 public long getDelay(TimeUnit unit) { long remaining = triggerTime - System.currentTimeMillis(); return unit.convert(remaining, TimeUnit.MILLISECONDS); // Returns decreasing value. Negative = ready now. } } // WRONG: Missing compareTo class BadTask implements Delayed { public long getDelay(TimeUnit unit) { /*...*/ } // Compiles fine. Throws ClassCastException when queue reorders. } // CORRECT: compareTo orders the internal heap class GoodTask implements Delayed { public long getDelay(TimeUnit unit) { /*...*/ } @Override public int compareTo(Delayed other) { return Long.compare( this.getDelay(TimeUnit.NANOSECONDS), other.getDelay(TimeUnit.NANOSECONDS) ); // Earliest trigger time = highest priority } } // Negative delays are correct behavior public long getDelay(TimeUnit unit) { long remaining = triggerTime - System.currentTimeMillis(); return unit.convert(remaining, TimeUnit.MILLISECONDS); // Negative = trigger time passed. Task ready immediately. // Zero = trigger time is now. Task ready immediately. // Positive = trigger time in future. Task blocks. } Production Pattern: Circuit Breaker with Retry 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 public class CircuitBreakerRetry { private final DelayQueue\u0026lt;RetryTask\u0026gt; retryQueue; private final AtomicInteger failureCount = new AtomicInteger(0); private volatile boolean circuitOpen = false; public void execute(String request) { if (circuitOpen) { throw new CircuitOpenException(\u0026#34;Circuit is open\u0026#34;); } try { callExternalService(request); failureCount.set(0); // Reset on success } catch (Exception e) { int failures = failureCount.incrementAndGet(); if (failures \u0026gt;= 5) { circuitOpen = true; scheduleCircuitReset(); } // Schedule retry with exponential backoff long delayMs = (long) Math.pow(2, failures) * 1000; retryQueue.put(new RetryTask(request, delayMs, failures)); } } private void scheduleCircuitReset() { retryQueue.put(new RetryTask(\u0026#34;RESET_CIRCUIT\u0026#34;, 30000, 0)); } } 2. ReentrantLock: Fine-Grained Control The Problem: Multiple threads update a shared resource. synchronized blocks forever if the lock is held. You need timeouts, interruptibility, or fair queueing.\nReentrantLock provides explicit locking with timeout and interruption support. More verbose than synchronized, but you control when to give up waiting.\nKey mechanism: tryLock(timeout) returns false if lock isn\u0026rsquo;t acquired within the timeout. Thread doesn\u0026rsquo;t block indefinitely.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 import java.util.concurrent.locks.*; import java.nio.file.*; import java.util.concurrent.TimeUnit; public class ConfigurationManager { private final Lock configLock = new ReentrantLock(true); // fair lock private final Path configFile; private String cachedConfig; public ConfigurationManager(Path configFile) { this.configFile = configFile; } public void updateConfig(String newConfig) { // Try to acquire lock with timeout try { if (configLock.tryLock(5, TimeUnit.SECONDS)) { try { System.out.println(Thread.currentThread().getName() + \u0026#34; - Updating config...\u0026#34;); // Write to file Files.writeString(configFile, newConfig); // Update cache cachedConfig = newConfig; // Simulate some processing Thread.sleep(1000); System.out.println(Thread.currentThread().getName() + \u0026#34; - Config updated\u0026#34;); } finally { configLock.unlock(); // ALWAYS in finally block } } else { System.err.println(Thread.currentThread().getName() + \u0026#34; - Could not acquire lock within 5 seconds\u0026#34;); } } catch (InterruptedException e) { Thread.currentThread().interrupt(); System.err.println(Thread.currentThread().getName() + \u0026#34; - Interrupted while waiting for lock\u0026#34;); } catch (Exception e) { System.err.println(\u0026#34;Config update failed: \u0026#34; + e.getMessage()); } } public String readConfig() { configLock.lock(); try { if (cachedConfig == null) { cachedConfig = Files.readString(configFile); } return cachedConfig; } catch (Exception e) { return \u0026#34;Error reading config\u0026#34;; } finally { configLock.unlock(); } } // Advanced: Check lock status without blocking public boolean isConfigBeingUpdated() { return ((ReentrantLock) configLock).isLocked(); } } Why not synchronized?\nsynchronized blocks forever if the lock is held. No timeout. No interruption. No way to check if locked without blocking.\nReentrantLock adds:\ntryLock(timeout) - give up after N seconds lockInterruptibly() - thread can be interrupted while waiting Fair mode - threads acquire lock in request order (prevents starvation) isLocked() - check status without blocking Trade-offs What you gain:\nTimeout support. Fail fast instead of blocking forever. Interruptibility. Shutdown threads cleanly mid-wait. Fair mode. Prevent thread starvation under contention. Condition variables. Multiple wait queues per lock. What you pay:\nManual unlock. Forget finally block = permanent lock. More verbose. 5 lines vs 1 for synchronized. Deadlock risk unchanged. Lock ordering still matters. No compiler help. synchronized checks happen at bytecode level. synchronized vs ReentrantLock:\nFeature synchronized ReentrantLock Timeout ❌ ✅ tryLock(timeout) Interruptible ❌ ✅ lockInterruptibly() Fair mode ❌ ✅ new ReentrantLock(true) Try without blocking ❌ ✅ tryLock() Condition variables ❌ (wait/notify only) ✅ newCondition() Automatic unlock ✅ ❌ (must use finally) Performance Slightly better Slightly worse Simplicity ✅ ❌ Failure modes:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 // WRONG: Exception skips unlock lock.lock(); doWork(); // Throws IOException lock.unlock(); // Never executes. Lock held forever. // All other threads block forever on this lock. // CORRECT: finally guarantees unlock lock.lock(); try { doWork(); } finally { lock.unlock(); // Runs even if doWork() throws } // WRONG: Inconsistent lock order causes deadlock // Thread 1: transfer(accountA, accountB, 100) synchronized(accountA) { synchronized(accountB) { /* transfer */ } } // Thread 2: transfer(accountB, accountA, 50) synchronized(accountB) { synchronized(accountA) { /* transfer */ } } // Thread 1 holds A, waits for B // Thread 2 holds B, waits for A // Deadlock. Both threads blocked forever. // CORRECT: Always acquire locks in same order Lock first = getLockById(Math.min(idA, idB)); Lock second = getLockById(Math.max(idA, idB)); first.lock(); try { second.lock(); try { doWork(); } finally { second.unlock(); } } finally { first.unlock(); } // Both threads acquire locks in ascending ID order. No circular wait. // WRONG: tryLock without finally if (lock.tryLock()) { doWork(); // Throws exception lock.unlock(); // Never executes } // Lock held forever // CORRECT: tryLock with finally if (lock.tryLock()) { try { doWork(); } finally { lock.unlock(); // Runs even if doWork() throws } } else { // Lock was held. Handle failure. } // WRONG: Ignoring interruption lock.lockInterruptibly(); // Throws InterruptedException - must catch // CORRECT: Restore interrupt status try { lock.lockInterruptibly(); try { doWork(); } finally { lock.unlock(); } } catch (InterruptedException e) { Thread.currentThread().interrupt(); // Restore flag // Thread was interrupted. Exit gracefully. } Advanced Pattern: Read-Write Lock 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 public class CachedData { private final ReadWriteLock rwLock = new ReentrantReadWriteLock(); private final Lock readLock = rwLock.readLock(); private final Lock writeLock = rwLock.writeLock(); private Map\u0026lt;String, String\u0026gt; cache = new HashMap\u0026lt;\u0026gt;(); public String read(String key) { readLock.lock(); // Multiple readers allowed try { return cache.get(key); } finally { readLock.unlock(); } } public void write(String key, String value) { writeLock.lock(); // Exclusive access try { cache.put(key, value); } finally { writeLock.unlock(); } } // Upgrade pattern: read -\u0026gt; write (tricky!) public void updateIfNeeded(String key, String newValue) { readLock.lock(); try { String current = cache.get(key); if (needsUpdate(current, newValue)) { // Release read lock before acquiring write lock readLock.unlock(); writeLock.lock(); try { // Re-check after acquiring write lock current = cache.get(key); if (needsUpdate(current, newValue)) { cache.put(key, newValue); } // Downgrade: acquire read lock before releasing write readLock.lock(); } finally { writeLock.unlock(); } } } finally { readLock.unlock(); } } } 3. Phaser: Dynamic Multi-Phase Coordination The Problem: ETL pipeline with Extract → Transform → Load phases. Workers join when data arrives. Workers leave when their partition completes. Party count changes during execution.\nCountDownLatch: Fixed count at creation. Cannot add parties. CyclicBarrier: Fixed parties. Cannot add or remove. Phaser: Parties register and deregister during execution.\nHow it works: Each worker calls register() on arrival and arriveAndDeregister() on completion. Phaser tracks current party count. Phase advances when all registered parties arrive.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 import java.util.concurrent.*; import java.util.stream.IntStream; public class ETLPipeline { static class ETLWorker implements Runnable { private final Phaser phaser; private final int workerId; public ETLWorker(Phaser phaser, int workerId) { this.phaser = phaser; this.workerId = workerId; phaser.register(); // Register this worker } @Override public void run() { try { System.out.println(\u0026#34;Worker-\u0026#34; + workerId + \u0026#34; - Starting EXTRACT phase\u0026#34;); Thread.sleep(ThreadLocalRandom.current().nextInt(500, 1500)); System.out.println(\u0026#34;Worker-\u0026#34; + workerId + \u0026#34; - EXTRACT complete\u0026#34;); phaser.arriveAndAwaitAdvance(); // Wait for all workers to finish extract System.out.println(\u0026#34;Worker-\u0026#34; + workerId + \u0026#34; - Starting TRANSFORM phase\u0026#34;); Thread.sleep(ThreadLocalRandom.current().nextInt(500, 1500)); System.out.println(\u0026#34;Worker-\u0026#34; + workerId + \u0026#34; - TRANSFORM complete\u0026#34;); phaser.arriveAndAwaitAdvance(); // Wait for all workers to finish transform System.out.println(\u0026#34;Worker-\u0026#34; + workerId + \u0026#34; - Starting LOAD phase\u0026#34;); Thread.sleep(ThreadLocalRandom.current().nextInt(500, 1500)); System.out.println(\u0026#34;Worker-\u0026#34; + workerId + \u0026#34; - LOAD complete\u0026#34;); phaser.arriveAndDeregister(); // Complete and leave } catch (InterruptedException e) { Thread.currentThread().interrupt(); } } } public void runPipeline() throws InterruptedException { // Create phaser with 1 party (main thread) Phaser phaser = new Phaser(1) { @Override protected boolean onAdvance(int phase, int registeredParties) { System.out.println(\u0026#34;\\n=== Phase \u0026#34; + phase + \u0026#34; completed. \u0026#34; + registeredParties + \u0026#34; parties registered ===\\n\u0026#34;); return false; // Continue to next phase } }; ExecutorService pool = Executors.newFixedThreadPool(4); // Start initial workers IntStream.range(0, 4).forEach(i -\u0026gt; pool.submit(new ETLWorker(phaser, i)) ); // Simulate dynamic worker joining mid-process Thread.sleep(2000); System.out.println(\u0026#34;\\n\u0026gt;\u0026gt;\u0026gt; NEW WORKER JOINING \u0026lt;\u0026lt;\u0026lt;\\n\u0026#34;); pool.submit(new ETLWorker(phaser, 99)); // Main thread deregisters phaser.arriveAndDeregister(); pool.shutdown(); pool.awaitTermination(1, TimeUnit.MINUTES); System.out.println(\u0026#34;\\nETL Pipeline complete!\u0026#34;); } } When to use Phaser:\nParties join/leave during execution More than 2 phases Need phase number tracking When NOT to use:\nFixed party count → Use CountDownLatch or CyclicBarrier Single phase → Use CountDownLatch Adds complexity. Only use when simpler tools don\u0026rsquo;t fit. Trade-offs What you gain:\nDynamic parties. Add workers mid-execution. Remove when done. Unlimited phases. CountDownLatch is one-shot. CyclicBarrier resets but doesn\u0026rsquo;t track phase number. Phase callbacks. onAdvance() runs between phases. Termination control. Return true from onAdvance() to stop. What you pay:\nComplexity. Hardest synchronizer to reason about. Register/deregister bugs. Forget to deregister = phase never advances. Higher overhead. Atomic operations on every arrive() call. Debugging difficulty. Thread dump shows \u0026ldquo;waiting on Phaser\u0026rdquo; but not which phase. Failure modes:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 // WRONG: Arrive without registering Phaser phaser = new Phaser(1); pool.submit(() -\u0026gt; { phaser.arriveAndAwaitAdvance(); // IllegalStateException // Thread never registered. Phaser doesn\u0026#39;t know about it. }); // CORRECT: Register before arriving class Worker implements Runnable { private final Phaser phaser; Worker(Phaser phaser) { this.phaser = phaser; phaser.register(); // Increment party count } public void run() { phaser.arriveAndAwaitAdvance(); // Now valid } } // WRONG: Register but never deregister phaser.register(); // Party count = 2 doWork(); return; // Exit without deregistering // Party count still 2. Other threads wait forever for this thread. // CORRECT: Deregister in finally phaser.register(); try { doWork(); phaser.arriveAndAwaitAdvance(); } finally { phaser.arriveAndDeregister(); // Decrement party count } // MISTAKE 3: onAdvance returns true too early Phaser phaser = new Phaser(1) { @Override protected boolean onAdvance(int phase, int registeredParties) { return true; // Terminates immediately! } }; // CORRECT: Return true only when done protected boolean onAdvance(int phase, int registeredParties) { System.out.println(\u0026#34;Phase \u0026#34; + phase + \u0026#34; complete\u0026#34;); return phase \u0026gt;= 2 || registeredParties == 0; // Stop after phase 2 } Phaser vs CountDownLatch vs CyclicBarrier:\nFeature CountDownLatch CyclicBarrier Phaser Reusable ❌ ✅ ✅ Dynamic parties ❌ ❌ ✅ Multiple phases ❌ ✅ ✅ Phase tracking ❌ ❌ ✅ Complexity Low Low High Use case One-shot wait Fixed multi-phase Dynamic multi-phase Debugging Concurrent Code Reading Thread Dumps Application hangs. No error logs. No exceptions. Take a thread dump—it shows every thread\u0026rsquo;s current state and stack trace.\nWindows:\n1 2 3 4 5 6 7 # Get PID jps # Generate thread dump jstack \u0026lt;PID\u0026gt; \u0026gt; thread_dump.txt # Or send Ctrl+Break to console Linux:\n1 kill -3 \u0026lt;PID\u0026gt; # Sends SIGQUIT, creates thread dump Example thread dump entry:\n\u0026#34;payment-worker-1\u0026#34; #23 prio=5 os_prio=0 tid=0x00007f8a2c001000 nid=0x4d2 waiting on condition\rjava.lang.Thread.State: WAITING (parking)\rat sun.misc.Unsafe.park(Native Method)\rat java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)\rat java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt\rat java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)\rat com.example.Service.process(Service.java:45) What this tells you:\nThread state: WAITING means blocked on a synchronizer (CountDownLatch, Semaphore, Lock) Stack trace: Shows where thread is stuck (line 45 in Service.java) Lock info: Which object thread is waiting on Red flags:\nMany threads WAITING on same object → Someone forgot countDown() or release() BLOCKED threads with circular dependencies → Deadlock Threads named \u0026ldquo;pool-1-thread-X\u0026rdquo; → No custom ThreadFactory (debugging nightmare) Thread in RUNNABLE for minutes → Infinite loop or expensive computation Common Deadlock Patterns Pattern 1: Lock Ordering Deadlock Why it deadlocks: Thread 1 holds lock A, needs lock B. Thread 2 holds lock B, needs lock A. Neither can proceed.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 // DEADLOCK class Account { private final Object lock = new Object(); public void transfer(Account to, double amount) { synchronized (this.lock) { // Thread 1 locks A synchronized (to.lock) { // Thread 1 waits for B this.balance -= amount; to.balance += amount; } } } } // Thread 1: accountA.transfer(accountB, 100); // Locks A, waits for B // Thread 2: accountB.transfer(accountA, 50); // Locks B, waits for A // Both threads wait forever. // FIX: Always acquire locks in same order // Use object hash code to determine order public void transfer(Account to, double amount) { Account first = System.identityHashCode(this) \u0026lt; System.identityHashCode(to) ? this : to; Account second = first == this ? to : this; synchronized (first.lock) { synchronized (second.lock) { if (this != first) { to.balance += amount; this.balance -= amount; } else { this.balance -= amount; to.balance += amount; } } } } // Now both threads acquire locks in same order. No circular wait. Pattern 2: Nested Lock Acquisition Why it deadlocks: method1() acquires lock1 → lock2 method2() acquires lock2 → lock1 Opposite order = deadlock\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 // DEADLOCK class Service { private final Lock lock1 = new ReentrantLock(); private final Lock lock2 = new ReentrantLock(); public void method1() { lock1.lock(); try { lock2.lock(); // Acquires lock2 while holding lock1 try { doWork(); } finally { lock2.unlock(); } } finally { lock1.unlock(); } } public void method2() { lock2.lock(); try { lock1.lock(); // Opposite order! try { doWork(); } finally { lock1.unlock(); } } finally { lock2.unlock(); } } } // FIX: Use tryLock with timeout // If can\u0026#39;t acquire second lock, release first and retry public void method2() { lock2.lock(); try { if (lock1.tryLock(1, TimeUnit.SECONDS)) { try { doWork(); } finally { lock1.unlock(); } } else { // Failed to acquire lock1 - back off throw new TimeoutException(\u0026#34;Could not acquire lock\u0026#34;); } } finally { lock2.unlock(); } } Pattern 3: Forgotten CountDownLatch Why it hangs: Latch expects 3 countdowns. Task only counts down on success. If task fails, count stays at 3. await() blocks forever.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 // HANGS FOREVER CountDownLatch latch = new CountDownLatch(3); pool.submit(() -\u0026gt; { doWork(); if (success) { // BUG: Conditional countdown latch.countDown(); } }); latch.await(); // Waits forever if doWork() fails // FIX: Always countDown, regardless of success pool.submit(() -\u0026gt; { try { doWork(); } finally { latch.countDown(); // Executes even if doWork() throws } }); Detecting Deadlocks Programmatically 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 public class DeadlockDetector { public static void detectDeadlocks() { ThreadMXBean threadBean = ManagementFactory.getThreadMXBean(); long[] deadlockedThreads = threadBean.findDeadlockedThreads(); if (deadlockedThreads != null) { ThreadInfo[] threadInfos = threadBean.getThreadInfo(deadlockedThreads); System.err.println(\u0026#34;DEADLOCK DETECTED!\u0026#34;); for (ThreadInfo info : threadInfos) { System.err.println(\u0026#34;Thread: \u0026#34; + info.getThreadName()); System.err.println(\u0026#34;Locked on: \u0026#34; + info.getLockName()); System.err.println(\u0026#34;Waiting for: \u0026#34; + info.getLockOwnerName()); System.err.println(\u0026#34;Stack trace:\u0026#34;); for (StackTraceElement element : info.getStackTrace()) { System.err.println(\u0026#34; \u0026#34; + element); } } // Alert monitoring system alertOps(\u0026#34;Deadlock detected!\u0026#34;); } } } Monitoring Concurrent Systems Why monitor: Thread pools fail silently. Queue fills up. Threads deadlock. No exceptions thrown. Metrics reveal problems before users notice.\nKey Metrics to Track 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 public class ConcurrencyMetrics { private final MeterRegistry registry; public void setupMetrics(ExecutorService pool, BlockingQueue\u0026lt;?\u0026gt; queue) { // Thread pool metrics if (pool instanceof ThreadPoolExecutor) { ThreadPoolExecutor tpe = (ThreadPoolExecutor) pool; registry.gauge(\u0026#34;threadpool.active\u0026#34;, tpe, ThreadPoolExecutor::getActiveCount); registry.gauge(\u0026#34;threadpool.size\u0026#34;, tpe, ThreadPoolExecutor::getPoolSize); registry.gauge(\u0026#34;threadpool.queue.size\u0026#34;, tpe, e -\u0026gt; e.getQueue().size()); // Alert if queue is growing registry.gauge(\u0026#34;threadpool.queue.utilization\u0026#34;, tpe, e -\u0026gt; { int queueSize = e.getQueue().size(); int remainingCapacity = e.getQueue().remainingCapacity(); return (double) queueSize / (queueSize + remainingCapacity); }); } // Task execution metrics registry.timer(\u0026#34;task.execution.time\u0026#34;); registry.counter(\u0026#34;task.completed\u0026#34;); registry.counter(\u0026#34;task.failed\u0026#34;); // Liveness check registry.gauge(\u0026#34;system.deadlocked.threads\u0026#34;, this, m -\u0026gt; countDeadlockedThreads()); } private long countDeadlockedThreads() { ThreadMXBean threadBean = ManagementFactory.getThreadMXBean(); long[] deadlocked = threadBean.findDeadlockedThreads(); return deadlocked != null ? deadlocked.length : 0; } } Critical Alerts Alert on:\nQueue depth \u0026gt; 80% capacity\nWhat it means: Producers add tasks faster than consumers process them.\nWhat breaks: Queue fills. New tasks rejected or block forever.\n1 2 3 if (queueUtilization \u0026gt; 0.8) { alert(\u0026#34;Queue nearly full - scale consumers\u0026#34;); } All threads busy for \u0026gt; 1 minute\nWhat it means: Every thread in pool is working. No idle threads.\nWhat breaks: New tasks queue up. Latency increases.\n1 2 3 if (activeThreads == poolSize \u0026amp;\u0026amp; duration \u0026gt; 60_000) { alert(\u0026#34;Thread pool saturated - increase size\u0026#34;); } Task duration p99 \u0026gt; SLA\nWhat it means: 99th percentile task takes longer than expected.\nWhat breaks: User-facing timeouts. SLA violations.\n1 2 3 if (taskDurationP99 \u0026gt; slaMs) { alert(\u0026#34;Task latency exceeds SLA\u0026#34;); } Deadlock detected\nWhat it means: Threads waiting on each other in a cycle.\nWhat breaks: Application hangs. Requires restart.\n1 2 3 if (deadlockedThreads \u0026gt; 0) { alert(\u0026#34;CRITICAL: Deadlock - restart required\u0026#34;); } Thread count growing\nWhat it means: New threads created but never destroyed.\nWhat breaks: Eventually hits OS thread limit. OutOfMemoryError.\n1 2 3 if (threadCountDelta \u0026gt; 10 \u0026amp;\u0026amp; timeWindow \u0026lt; 60_000) { alert(\u0026#34;Thread leak detected\u0026#34;); } Common Anti-Patterns Across All Tools 1. Thread Pool Sizing 1 2 3 4 5 6 7 8 9 10 // ANTI-PATTERN: Magic numbers ExecutorService pool = Executors.newFixedThreadPool(42); // CORRECT: Calculate based on workload int poolSize = Runtime.getRuntime().availableProcessors() * (isCpuBound ? 1 : 2); // BETTER: Formula for I/O-bound // ThreadCount = NumCores * (1 + WaitTime/ComputeTime) int poolSize = cores * (1 + (int)(waitTimeMs / computeTimeMs)); 2. Ignoring Shutdown What breaks: Threads keep running after method returns. JVM doesn\u0026rsquo;t exit. Application hangs on shutdown.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 // WRONG: No shutdown public void process() { ExecutorService pool = Executors.newFixedThreadPool(10); pool.submit(() -\u0026gt; work()); } // Threads still running. Pool never cleaned up. // CORRECT: Shutdown sequence public void process() { ExecutorService pool = Executors.newFixedThreadPool(10); try { pool.submit(() -\u0026gt; work()).get(); } finally { pool.shutdown(); // Stop accepting new tasks if (!pool.awaitTermination(60, TimeUnit.SECONDS)) { pool.shutdownNow(); // Force stop running tasks } } } 3. Daemon Threads for Critical Work What breaks: JVM exits while daemon thread is mid-write. Database transaction incomplete. Data corrupted.\nThe rule: Daemon threads die when JVM exits. Non-daemon threads keep JVM alive until they finish.\n1 2 3 4 5 6 7 8 9 10 11 12 13 // WRONG: Daemon thread for DB writes ThreadFactory factory = r -\u0026gt; { Thread t = new Thread(r); t.setDaemon(true); // JVM exits → thread dies → DB write lost return t; }; // CORRECT: Non-daemon for critical work ThreadFactory factory = r -\u0026gt; { Thread t = new Thread(r); t.setDaemon(false); // JVM waits for thread to finish return t; }; 4. Unbounded Queues What breaks: Producer faster than consumer. Queue grows. Heap fills. OutOfMemoryError.\n1 2 3 4 5 6 // WRONG: Unbounded queue BlockingQueue\u0026lt;Task\u0026gt; queue = new LinkedBlockingQueue\u0026lt;\u0026gt;(); // No limit! // CORRECT: Bounded queue BlockingQueue\u0026lt;Task\u0026gt; queue = new ArrayBlockingQueue\u0026lt;\u0026gt;(1000); // When full, put() blocks. Producer slows down. Backpressure applied. 5. Swallowing Exceptions What breaks: Task throws exception. submit() catches it silently. No logs. No alerts. Silent failure.\nWhy it happens: submit() returns Future. Exception stored in Future. If you don\u0026rsquo;t call get(), you never see it.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 // WRONG: Exception disappears pool.submit(() -\u0026gt; { riskyOperation(); // Throws exception → stored in Future → never retrieved }); // CORRECT: Explicit error handling pool.submit(() -\u0026gt; { try { riskyOperation(); } catch (Exception e) { logger.error(\u0026#34;Task failed\u0026#34;, e); errorTracker.captureException(e); } }); Integration Example: Complete Concurrent Pipeline Combining multiple tools to build a realistic image processing pipeline:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 public class ProductionImagePipeline { private final ExecutorService downloadPool; private final ExecutorService processPool; private final BlockingQueue\u0026lt;ImageTask\u0026gt; downloadQueue; private final BlockingQueue\u0026lt;ImageTask\u0026gt; processQueue; private final Semaphore rateLimiter; private final MeterRegistry metrics; public ProductionImagePipeline(MeterRegistry metrics) { this.metrics = metrics; // Custom thread factories for debugging this.downloadPool = Executors.newFixedThreadPool(10, new NamedThreadFactory(\u0026#34;image-downloader\u0026#34;, false, Thread.NORM_PRIORITY)); this.processPool = Executors.newFixedThreadPool(4, new NamedThreadFactory(\u0026#34;image-processor\u0026#34;, false, Thread.NORM_PRIORITY)); // Bounded queues for backpressure this.downloadQueue = new ArrayBlockingQueue\u0026lt;\u0026gt;(100); this.processQueue = new ArrayBlockingQueue\u0026lt;\u0026gt;(50); // Rate limiter: 10 downloads per second this.rateLimiter = new Semaphore(10, true); // Setup monitoring setupMetrics(); } public CompletableFuture\u0026lt;ProcessedImage\u0026gt; process(String imageUrl) { return CompletableFuture .supplyAsync(() -\u0026gt; downloadWithRetry(imageUrl), downloadPool) .thenApplyAsync(image -\u0026gt; processImage(image), processPool) .exceptionally(ex -\u0026gt; { metrics.counter(\u0026#34;pipeline.failed\u0026#34;).increment(); logger.error(\u0026#34;Pipeline failed for \u0026#34; + imageUrl, ex); return null; }); } private RawImage downloadWithRetry(String url) { // DelayQueue pattern for retry int attempt = 0; while (attempt \u0026lt; 3) { try { // Rate limiting with semaphore rateLimiter.acquire(); try { return download(url); } finally { rateLimiter.release(); } } catch (Exception e) { attempt++; if (attempt \u0026gt;= 3) throw e; long backoff = (long) Math.pow(2, attempt) * 1000; Thread.sleep(backoff); } } throw new RuntimeException(\u0026#34;Max retries exceeded\u0026#34;); } private void setupMetrics() { metrics.gauge(\u0026#34;queue.download.size\u0026#34;, downloadQueue, BlockingQueue::size); metrics.gauge(\u0026#34;queue.process.size\u0026#34;, processQueue, BlockingQueue::size); metrics.gauge(\u0026#34;ratelimiter.available\u0026#34;, rateLimiter, Semaphore::availablePermits); } public void shutdown() { downloadPool.shutdown(); processPool.shutdown(); try { downloadPool.awaitTermination(60, TimeUnit.SECONDS); processPool.awaitTermination(60, TimeUnit.SECONDS); } catch (InterruptedException e) { downloadPool.shutdownNow(); processPool.shutdownNow(); } } } Production Readiness Checklist Before deploying concurrent code:\nResource Management All thread pools have bounded queues Shutdown handlers registered Timeouts on all blocking operations Thread pool sizes calculated, not guessed Observability Thread pools have meaningful names (ThreadFactory) Queue depths monitored Task execution times tracked Deadlock detection enabled Exception handlers log to monitoring system Error Handling All Future.get() calls have timeouts Async task exceptions logged Circuit breakers on external calls Retry logic with exponential backoff Testing Load tests with concurrent users Chaos engineering (kill threads mid-execution) Thread dump analysis under load Memory profiling for leaks Complete Toolkit Summary (All 3 Parts) Execution Tools (Part 1 - Day 95) Executor → Submit and forget tasks ExecutorService → Managed thread pools ScheduledExecutorService → Periodic execution Future → Blocking result retrieval CompletableFuture → Non-blocking async pipelines CountDownLatch → One-shot coordination Synchronization Tools (Part 2 - Day 96) CyclicBarrier → Multi-phase checkpoints Semaphore → Resource limiting ThreadFactory → Thread customization BlockingQueue → Producer-consumer Advanced Tools (Part 3 - Today) DelayQueue → Delayed execution ReentrantLock → Custom locking Phaser → Dynamic multi-phase Choosing the Right Tool: Decision Tree From my understanding we can have draw this decision tree as our mental model where we can have some idea to choose which tool will go with whic types of task the best.\nKey Principles I\u0026rsquo;ve Learned 1. Always Set Timeouts Future.get() without timeout blocks forever if task hangs. Lock.lock() without timeout deadlocks permanently. Always use timeout variants: get(timeout), tryLock(timeout), acquire(timeout).\n2. Name Your Threads Thread dump shows \u0026ldquo;pool-1-thread-17\u0026rdquo;. Which service? Which operation? Unknown. Custom ThreadFactory adds meaningful names: \u0026ldquo;payment-processor-3\u0026rdquo;. Debugging becomes possible.\n3. Monitor Queue Depths Queue at 10% → normal. Queue at 90% → consumers falling behind. Queue at 100% → producers blocking. Monitor queue depth. Alert before it fills.\n4. Graceful Shutdown is Hard Three steps: shutdown() stops accepting new tasks. awaitTermination(timeout) waits for running tasks. shutdownNow() interrupts if timeout exceeded. Miss any step → threads leak or work lost.\n5. Beware of Daemon Threads Daemon thread flushing logs. JVM exits. Thread dies mid-write. Logs lost. Use daemon threads only for non-critical background work.\n6. Fair vs Unfair Locks Matter Fair lock: threads acquire in request order. Prevents starvation. Lower throughput. Unfair lock: any thread can acquire. Higher throughput. Starvation possible. Default is unfair. Choose explicitly.\n7. Bounded Resources Prevent Cascading Failures Unbounded queue fills memory. OOM kills application. Bounded queue applies backpressure. Producer slows down. System stays alive.\n8. Composition Over Complexity Need one-shot coordination? CountDownLatch. Need reusable barrier? CyclicBarrier. Need dynamic parties? Then Phaser. Start simple. Add complexity only when required.\nWhen NOT to Use These Tools Before reaching for concurrency, ask:\nDon\u0026rsquo;t use if:\nSequential processing takes \u0026lt; 100ms Task is sequential (B depends on A) Dataset is small (\u0026lt; 1000 items) Debugging complexity outweighs performance gain Team lacks concurrency expertise Do use if:\nI/O-bound operations (network, disk, DB) Independent tasks that parallelize Need to keep UI/API responsive Processing large datasets Proven bottleneck exists Rule: Measure first. Prove the bottleneck. Then add concurrency. Concurrency adds complexity. Complexity adds bugs. Only pay that cost when the benefit is real.\nUnderstanding the Complete Toolkit Over these three parts, we\u0026rsquo;ve explored 13 essential concurrency tools. I\u0026rsquo;ve used some extensively in production (ExecutorService, Future, CountDownLatch, ReentrantLock), while others (Phaser, DelayQueue) I\u0026rsquo;m still mastering.\nThe insight: Concurrency isn\u0026rsquo;t about speed. It\u0026rsquo;s about structure. How do you coordinate independent work? How do you prevent threads from corrupting shared state? How do you fail gracefully under load?\nEach tool solves a specific problem:\nExecute? → ExecutorService Wait? → CountDownLatch / CyclicBarrier / Phaser Limit? → Semaphore Pass work? → BlockingQueue Delay? → DelayQueue / ScheduledExecutorService Custom locking? → ReentrantLock The skill is knowing which tool fits your use case. That\u0026rsquo;s what these 13 tools teach.\nWhat\u0026rsquo;s Next? This completes our deep dive into java.util.concurrent. More to learn:\nVirtual Threads (Java 19+) - Lightweight threads that change the concurrency model Reactive Streams - Async streams with backpressure (Project Reactor, RxJava) Fork/Join Framework - Recursive parallelism for divide-and-conquer Parallel Streams - When and when not to use them Structured Concurrency (Preview) - Better async task management Next time you\u0026rsquo;re tempted to spawn a raw Thread, ask yourself: which of these 13 patterns fits my use case?\nThat\u0026rsquo;s the question I\u0026rsquo;m learning to answer.\nPrevious: Day 96 - Java Concurrency Toolkit Part 2: Core Synchronization Patterns\n","permalink":"https://mohibul.pages.dev/posts/posts/java/100daysofjava/day97/","summary":"Part 3: Advanced Patterns \u0026amp; Production Readiness - Deep dive into 3 advanced tools plus debugging, monitoring, and going to production","title":"Day 97: Java Concurrency Toolkit Part 3 - Advanced Patterns \u0026 Production Readiness"},{"content":" Series Navigation: Part 1: Foundation \u0026amp; Execution • Part 2 (You are here) • Part 3: Advanced Patterns\n\u0026ldquo;The art of concurrent programming lies not in making things parallel, but in coordinating parallel things gracefully.\u0026rdquo;\nIn Part 1 (Day 95), we covered foundation and execution patterns: Executor, ExecutorService, ScheduledExecutorService, Future, CompletableFuture, and CountDownLatch. These tools execute tasks concurrently.\nPart 2 covers core synchronization patterns—the tools that coordinate threads and protect shared resources.\nSeries Overview This is a 3-part series on Java Concurrency:\nPart 1 (Day 95): Foundation \u0026amp; Execution\nExecutor, ExecutorService, ScheduledExecutorService Future, CompletableFuture, CountDownLatch Part 2 (Today - Day 96): Core Synchronization Patterns\nCyclicBarrier, Semaphore, ThreadFactory, BlockingQueue Part 3 (Day 97): Advanced Patterns \u0026amp; Production Readiness\nDelayQueue, ReentrantLock, Phaser Debugging, Monitoring, Production Patterns What We\u0026rsquo;re Covering Today Part 2 (Today): Core Synchronization Patterns\nCyclicBarrier - Reusable multi-phase synchronization Semaphore - Resource pool management ThreadFactory - Production-ready thread management BlockingQueue - Producer-consumer patterns Quick Reference: Part 2 Tools at a Glance Tool TL;DR Caveat CyclicBarrier Reusable synchronization checkpoint for fixed threads. All must arrive before any proceed. Fixed party count. If one thread fails, all block forever Semaphore Controls concurrent access to limited resources. Think \u0026ldquo;permission slips\u0026rdquo; (N max). Must release in finally - easy to leak permits ThreadFactory Customizes thread creation. Name your threads for debugging! Daemon threads die when JVM exits - bad for critical work BlockingQueue Decouples producers/consumers with thread-safe queue. Built-in backpressure. Unbounded queues = OOM. Use bounded ArrayBlockingQueue The Coordination Challenge Part 1 showed how to execute work concurrently. But execution alone isn\u0026rsquo;t enough. Real systems need coordination:\nThink of it like a construction project:\nWorkers need to sync at checkpoints (CyclicBarrier) Only X workers can use the crane at once (Semaphore) Workers need proper uniforms and tools (ThreadFactory) Materials move between stations (BlockingQueue) 1. CyclicBarrier: Multi-Phase Processing The Problem: You\u0026rsquo;re processing a large dataset in three phases: Extract → Transform → Load. Each phase must finish across all threads before the next begins.\nUse CyclicBarrier for batch processing with distinct phases. Unlike CountDownLatch (one-shot), CyclicBarrier resets after each phase. This makes iterative processing simple.\nThe Solution: Synchronized phase transitions with CyclicBarrier.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 import java.util.concurrent.*; import java.util.stream.IntStream; public class BatchProcessor { private final int workerCount; public BatchProcessor(int workerCount) { this.workerCount = workerCount; } public void processBatch() throws InterruptedException { CyclicBarrier barrier = new CyclicBarrier( workerCount, () -\u0026gt; System.out.println(\u0026#34;--- Phase completed. All workers synchronized ---\u0026#34;) ); ExecutorService pool = Executors.newFixedThreadPool(workerCount); Runnable worker = () -\u0026gt; { try { String threadName = Thread.currentThread().getName(); // Phase 1: Extract System.out.println(threadName + \u0026#34; - Extracting data...\u0026#34;); Thread.sleep(ThreadLocalRandom.current().nextInt(500, 1500)); System.out.println(threadName + \u0026#34; - Extract complete\u0026#34;); barrier.await(); // Wait for all threads to finish Phase 1 // Phase 2: Transform System.out.println(threadName + \u0026#34; - Transforming data...\u0026#34;); Thread.sleep(ThreadLocalRandom.current().nextInt(500, 1500)); System.out.println(threadName + \u0026#34; - Transform complete\u0026#34;); barrier.await(); // Wait for all threads to finish Phase 2 // Phase 3: Load System.out.println(threadName + \u0026#34; - Loading data...\u0026#34;); Thread.sleep(ThreadLocalRandom.current().nextInt(500, 1500)); System.out.println(threadName + \u0026#34; - Load complete\u0026#34;); barrier.await(); // Wait for all threads to finish Phase 3 } catch (InterruptedException | BrokenBarrierException e) { System.err.println(\u0026#34;Worker interrupted: \u0026#34; + e.getMessage()); } }; // Submit all workers IntStream.range(0, workerCount) .forEach(i -\u0026gt; pool.submit(worker)); pool.shutdown(); pool.awaitTermination(1, TimeUnit.MINUTES); System.out.println(\u0026#34;Batch processing complete!\u0026#34;); } } Key Difference from CountDownLatch: CyclicBarrier is reusable. After all threads reach the barrier, it resets automatically. Use it for iterative or phase-based processing.\nVisualizing CyclicBarrier Here\u0026rsquo;s a sequence diagram showing how CyclicBarrier works:\nTrade-offs and Limitations Pros:\nReusable (unlike CountDownLatch) Built-in barrier action (callback when all arrive) Low overhead Thread-safe reset Cons:\nFixed number of parties (must be known at creation) If one thread fails, others wait forever (unless you handle BrokenBarrierException) No dynamic party registration (use Phaser instead) All threads must reach barrier together (can\u0026rsquo;t have stragglers) Common Mistakes:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 // MISTAKE 1: Wrong party count CyclicBarrier barrier = new CyclicBarrier(5); for (int i = 0; i \u0026lt; 3; i++) { // Only 3 threads! pool.submit(() -\u0026gt; { doWork(); barrier.await(); // Blocks forever! Only 3/5 arrived }); } // CORRECT: Match party count int parties = 3; CyclicBarrier barrier = new CyclicBarrier(parties); for (int i = 0; i \u0026lt; parties; i++) { pool.submit(() -\u0026gt; { doWork(); barrier.await(); }); } // MISTAKE 2: Not handling BrokenBarrierException pool.submit(() -\u0026gt; { doWork(); barrier.await(); // What if another thread throws exception? }); // CORRECT: Handle broken barrier pool.submit(() -\u0026gt; { try { doWork(); barrier.await(); } catch (BrokenBarrierException e) { System.err.println(\u0026#34;Barrier broken! Aborting.\u0026#34;); // Cleanup and exit } catch (InterruptedException e) { Thread.currentThread().interrupt(); } }); // MISTAKE 3: Reusing after reset without checking if (barrier.isBroken()) { barrier.reset(); // Must reset explicitly if broken } // CORRECT: Check state before reuse if (barrier.isBroken()) { barrier.reset(); } barrier.await(); When to Use:\nFixed number of worker threads Multi-phase processing where all must complete each phase Iterative algorithms (simulations, game loops) Don\u0026rsquo;t use when:\nYou need dynamic party count (use Phaser) You need one-shot synchronization (use CountDownLatch) 2. Semaphore: Resource Pool Management The Problem: Your database connection pool has 10 connections. If 50 threads query simultaneously, you overwhelm the pool and get timeouts.\nSemaphore limits concurrent access to finite resources. Acquire a permit before using the resource, release it when done.\nThe Solution: Limit concurrent access with Semaphore.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 import java.util.concurrent.*; public class DatabaseService { private final Semaphore connectionPermits; private final int maxConnections; public DatabaseService(int maxConnections) { this.maxConnections = maxConnections; this.connectionPermits = new Semaphore(maxConnections, true); // fair=true for FIFO } public String executeQuery(String sql) { System.out.println(Thread.currentThread().getName() + \u0026#34; - Requesting DB connection...\u0026#34;); try { // Try to acquire permit (blocks if none available) connectionPermits.acquire(); System.out.println(Thread.currentThread().getName() + \u0026#34; - Got connection! \u0026#34; + \u0026#34;Available: \u0026#34; + connectionPermits.availablePermits() + \u0026#34;/\u0026#34; + maxConnections); // Execute query String result = runActualQuery(sql); return result; } catch (InterruptedException e) { Thread.currentThread().interrupt(); return \u0026#34;Query interrupted\u0026#34;; } finally { // Always release permit connectionPermits.release(); System.out.println(Thread.currentThread().getName() + \u0026#34; - Released connection\u0026#34;); } } public boolean tryExecuteQueryWithTimeout(String sql, long timeoutMs) { try { // Try to acquire with timeout if (connectionPermits.tryAcquire(timeoutMs, TimeUnit.MILLISECONDS)) { try { runActualQuery(sql); return true; } finally { connectionPermits.release(); } } else { System.err.println(\u0026#34;Could not acquire connection within \u0026#34; + timeoutMs + \u0026#34;ms\u0026#34;); return false; } } catch (InterruptedException e) { Thread.currentThread().interrupt(); return false; } } private String runActualQuery(String sql) { // Simulate DB query try { Thread.sleep(1000); } catch (InterruptedException e) { Thread.currentThread().interrupt(); } return \u0026#34;Result for: \u0026#34; + sql; } } Pro Tip: Use tryAcquire() with timeout in request handlers to fail fast instead of blocking indefinitely. Your users will thank you.\nTrade-offs and Limitations Pros:\nSimple API (acquire/release) Supports fair vs non-fair modes Can try with timeout Can query available permits Very low overhead Cons:\nNo automatic resource management (must call release in finally) Permits can be released by any thread (not tied to acquirer) No built-in timeout enforcement on resource usage Can accidentally create permit leaks Common Mistakes:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 // MISTAKE 1: Forgetting to release semaphore.acquire(); doWork(); // What if this throws? semaphore.release(); // Never called! // CORRECT: Always use try-finally semaphore.acquire(); try { doWork(); } finally { semaphore.release(); // Always happens } // MISTAKE 2: Releasing without acquiring semaphore.release(); // Increases permit count above initial! // This creates \u0026#34;phantom permits\u0026#34; that break your resource limit // CORRECT: Only release what you acquired boolean acquired = semaphore.tryAcquire(); if (acquired) { try { doWork(); } finally { semaphore.release(); } } // MISTAKE 3: Using unfair mode for critical resources Semaphore semaphore = new Semaphore(1, false); // unfair // Threads can \u0026#34;cut in line\u0026#34; causing starvation // CORRECT: Use fair mode for critical resources Semaphore semaphore = new Semaphore(1, true); // fair = FIFO What is Fair/Unfair in a Semaphore I\u0026rsquo;ve mentioned fair/unfair mode in the code above. Here\u0026rsquo;s what they mean:\nSemaphore operates in either fair or unfair mode. This determines the order waiting threads acquire permits. You trade guaranteed access for throughput.\nFair Mode:\nThreads acquire permits in request order (FIFO queue) The longest-waiting thread gets the next available permit Prevents starvation—no thread waits forever while new threads keep acquiring permits Use for critical resources like database connection pools Unfair Mode:\nBetter performance—no FIFO queue overhead means fewer context switches Higher throughput but some threads might starve To create a fair semaphore, you use the constructor new Semaphore(int permits, true).\nReal-World Use Cases:\nDatabase connection pools API rate limiting (X requests per second) Limited worker threads for CPU-intensive tasks Bounded parallelism in file I/O Integration Example: Rate Limiting 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 public class RateLimiter { private final Semaphore permits; private final ScheduledExecutorService refiller; public RateLimiter(int permitsPerSecond) { this.permits = new Semaphore(permitsPerSecond, true); this.refiller = Executors.newScheduledThreadPool(1); // Refill permits every second refiller.scheduleAtFixedRate(() -\u0026gt; { int available = permits.availablePermits(); int toAdd = permitsPerSecond - available; if (toAdd \u0026gt; 0) { permits.release(toAdd); } }, 1, 1, TimeUnit.SECONDS); } public boolean tryAcquire() { return permits.tryAcquire(); } } 3. ThreadFactory: Production-Ready Thread Management The Problem: Your thread dump shows dozens of threads named pool-1-thread-X. You can\u0026rsquo;t debug which pool causes the issue.\nCustom ThreadFactory gives you control over thread creation. Name threads for easier debugging and monitoring.\nThe Solution: Custom ThreadFactory with meaningful names and proper configuration.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 import java.util.concurrent.*; import java.util.concurrent.atomic.AtomicInteger; public class NamedThreadFactory implements ThreadFactory { private final String namePrefix; private final AtomicInteger threadNumber; private final boolean daemon; private final int priority; public NamedThreadFactory(String namePrefix, boolean daemon, int priority) { this.namePrefix = namePrefix; this.daemon = daemon; this.priority = priority; this.threadNumber = new AtomicInteger(1); } @Override public Thread newThread(Runnable r) { Thread t = new Thread(r); t.setName(namePrefix + \u0026#34;-\u0026#34; + threadNumber.getAndIncrement()); t.setDaemon(daemon); t.setPriority(priority); // Add uncaught exception handler t.setUncaughtExceptionHandler((thread, throwable) -\u0026gt; { System.err.println(\u0026#34;Uncaught exception in \u0026#34; + thread.getName() + \u0026#34;: \u0026#34; + throwable.getMessage()); // In production: send to error tracking service like sentry, rollbar etc }); return t; } public static ExecutorService createNamedPool(String name, int size) { return Executors.newFixedThreadPool( size, new NamedThreadFactory(name, false, Thread.NORM_PRIORITY) ); } } // Usage example class ServiceOrchestrator { private final ExecutorService paymentPool; private final ExecutorService notificationPool; private final ExecutorService analyticsPool; public ServiceOrchestrator() { this.paymentPool = NamedThreadFactory.createNamedPool(\u0026#34;payment-worker\u0026#34;, 8); this.notificationPool = NamedThreadFactory.createNamedPool(\u0026#34;notification-worker\u0026#34;, 4); this.analyticsPool = NamedThreadFactory.createNamedPool(\u0026#34;analytics-worker\u0026#34;, 2); } // Now thread dumps show: payment-worker-1, notification-worker-1, analytics-worker-1 // Instead of: pool-1-thread-1, pool-2-thread-1, pool-3-thread-1 } Production Checklist:\nMeaningful thread names Daemon status set correctly Uncaught exception handlers Appropriate priority levels Trade-offs and Limitations Pros:\nFull control over thread properties Consistent naming for debugging Centralized exception handling Can add custom initialization logic Cons:\nMore boilerplate than default factories Easy to forget to use it Doesn\u0026rsquo;t prevent thread leaks (just makes them easier to find) Common Mistakes:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 // MISTAKE 1: Using daemon threads for critical work ThreadFactory factory = r -\u0026gt; { Thread t = new Thread(r); t.setDaemon(true); // JVM can exit while thread is running! return t; }; ExecutorService pool = Executors.newFixedThreadPool(10, factory); pool.submit(() -\u0026gt; { writeToDatabase(); // May not complete before JVM exits! }); // CORRECT: Non-daemon for critical work ThreadFactory factory = r -\u0026gt; { Thread t = new Thread(r); t.setDaemon(false); // JVM waits for completion return t; }; // MISTAKE 2: Not thread-safe counter class BadFactory implements ThreadFactory { private int counter = 0; // Not thread-safe! public Thread newThread(Runnable r) { return new Thread(r, \u0026#34;worker-\u0026#34; + counter++); } } // CORRECT: Use AtomicInteger class GoodFactory implements ThreadFactory { private final AtomicInteger counter = new AtomicInteger(0); public Thread newThread(Runnable r) { return new Thread(r, \u0026#34;worker-\u0026#34; + counter.getAndIncrement()); } } // MISTAKE 3: Swallowing exceptions in handler t.setUncaughtExceptionHandler((thread, throwable) -\u0026gt; { // Just print? Exception is lost! throwable.printStackTrace(); }); // CORRECT: Send to monitoring system t.setUncaughtExceptionHandler((thread, throwable) -\u0026gt; { logger.error(\u0026#34;Uncaught exception in \u0026#34; + thread.getName(), throwable); // Send to Sentry, Rollbar, etc. errorTracker.captureException(throwable); // Update metrics exceptionCounter.increment(); }); Monitoring Integration 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 public class MonitoredThreadFactory implements ThreadFactory { private final String namePrefix; private final AtomicInteger threadNumber = new AtomicInteger(1); private final AtomicInteger createdThreads; private final AtomicInteger activeThreads; public MonitoredThreadFactory(String namePrefix, MeterRegistry registry) { this.namePrefix = namePrefix; this.createdThreads = registry.gauge(\u0026#34;threads.created\u0026#34;, Tags.of(\u0026#34;pool\u0026#34;, namePrefix), new AtomicInteger(0)); this.activeThreads = registry.gauge(\u0026#34;threads.active\u0026#34;, Tags.of(\u0026#34;pool\u0026#34;, namePrefix), new AtomicInteger(0)); } @Override public Thread newThread(Runnable r) { createdThreads.incrementAndGet(); activeThreads.incrementAndGet(); Thread t = new Thread(() -\u0026gt; { try { r.run(); } finally { activeThreads.decrementAndGet(); } }); t.setName(namePrefix + \u0026#34;-\u0026#34; + threadNumber.getAndIncrement()); return t; } } 4. BlockingQueue: The Producer-Consumer Pattern The Problem: Your service receives 1000 requests/second but processes only 100/second. Without buffering, you drop requests or crash.\nBlockingQueue decouples work submission from execution. It provides natural backpressure when consumers can\u0026rsquo;t keep up.\nThe Solution: Decouple producers from consumers with a bounded queue.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 import java.util.concurrent.*; import java.nio.file.*; import java.util.stream.Stream; public class ImageProcessingPipeline { private final BlockingQueue\u0026lt;Path\u0026gt; imageQueue; private final ExecutorService producers; private final ExecutorService consumers; private volatile boolean running; public ImageProcessingPipeline(int queueSize, int consumerCount) { this.imageQueue = new ArrayBlockingQueue\u0026lt;\u0026gt;(queueSize); this.producers = Executors.newSingleThreadExecutor( new NamedThreadFactory(\u0026#34;image-producer\u0026#34;, false, Thread.NORM_PRIORITY) ); this.consumers = Executors.newFixedThreadPool( consumerCount, new NamedThreadFactory(\u0026#34;image-consumer\u0026#34;, false, Thread.NORM_PRIORITY) ); this.running = true; } public void start(Path imageDirectory) { System.out.println(\u0026#34;Starting image processing pipeline...\u0026#34;); // Producer: Read image files and add to queue producers.submit(() -\u0026gt; { try (Stream\u0026lt;Path\u0026gt; paths = Files.list(imageDirectory)) { paths.filter(p -\u0026gt; p.toString().endsWith(\u0026#34;.jpg\u0026#34;)) .forEach(path -\u0026gt; { try { System.out.println(\u0026#34;Producer: Queuing \u0026#34; + path.getFileName()); imageQueue.put(path); // Blocks if queue is full } catch (InterruptedException e) { Thread.currentThread().interrupt(); } }); // Signal completion by adding poison pills for (int i = 0; i \u0026lt; Runtime.getRuntime().availableProcessors(); i++) { imageQueue.put(Paths.get(\u0026#34;STOP\u0026#34;)); // Sentinel value } } catch (Exception e) { System.err.println(\u0026#34;Producer error: \u0026#34; + e.getMessage()); } }); // Consumers: Process images from queue int consumerCount = Runtime.getRuntime().availableProcessors(); for (int i = 0; i \u0026lt; consumerCount; i++) { consumers.submit(() -\u0026gt; { while (running) { try { Path image = imageQueue.take(); // Blocks if queue is empty if (image.toString().equals(\u0026#34;STOP\u0026#34;)) { System.out.println(\u0026#34;Consumer received stop signal\u0026#34;); break; } System.out.println(\u0026#34;Consumer: Processing \u0026#34; + image.getFileName()); compressImage(image); } catch (InterruptedException e) { Thread.currentThread().interrupt(); break; } } }); } } private void compressImage(Path image) { // Simulate compression work try { Thread.sleep(500); System.out.println(\u0026#34;Compressed: \u0026#34; + image.getFileName()); } catch (InterruptedException e) { Thread.currentThread().interrupt(); } } public void shutdown() { running = false; producers.shutdown(); consumers.shutdown(); } } Key Pattern: Use the \u0026ldquo;poison pill\u0026rdquo; technique—a sentinel value (STOP in this example) that signals consumers to stop gracefully.\nTrade-offs and Limitations Pros:\nBuilt-in backpressure (blocks when full) Thread-safe operations Natural decoupling of producers/consumers Multiple queue implementations (Array, Linked, Priority, etc.) Cons:\nBounded queues block producers when full Unbounded queues cause OOM No built-in monitoring for queue depth Poison pill pattern needs careful coordination Queue Implementation Trade-offs:\nImplementation Ordering Bounded Throughput Use Case ArrayBlockingQueue FIFO ✅ High Fixed capacity, predictable memory LinkedBlockingQueue FIFO Optional Medium Variable load, unbounded acceptable PriorityBlockingQueue Priority ❌ Lower Task prioritization needed SynchronousQueue N/A N/A Highest Direct handoff (zero capacity) DelayQueue Delay time ❌ Medium Delayed execution Common Mistakes:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 // MISTAKE 1: Unbounded queue with slow consumers BlockingQueue\u0026lt;Task\u0026gt; queue = new LinkedBlockingQueue\u0026lt;\u0026gt;(); // Unbounded! // Producer adds 1000 items/sec, consumer processes 10/sec // Queue grows to millions = OOM! // CORRECT: Bounded queue with backpressure BlockingQueue\u0026lt;Task\u0026gt; queue = new ArrayBlockingQueue\u0026lt;\u0026gt;(1000); // Producer blocks when queue is full = natural backpressure // MISTAKE 2: Not handling interruption queue.put(task); // Blocks if full, but what if interrupted? // CORRECT: Handle interruption try { queue.put(task); } catch (InterruptedException e) { Thread.currentThread().interrupt(); // Cleanup and exit gracefully } // MISTAKE 3: Poison pill per producer instead of per consumer // 2 producers, 3 consumers queue.put(STOP); // Only 1 consumer stops! // CORRECT: One poison pill per consumer int consumerCount = 3; for (int i = 0; i \u0026lt; consumerCount; i++) { queue.put(STOP); } // MISTAKE 4: Losing tasks on shutdown pool.shutdown(); // Queued tasks might not complete! // CORRECT: Drain queue before shutdown pool.shutdown(); List\u0026lt;Runnable\u0026gt; unfinished = new ArrayList\u0026lt;\u0026gt;(); queue.drainTo(unfinished); // Persist unfinished tasks for retry Monitoring Queue Health 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 public class MonitoredQueue\u0026lt;T\u0026gt; { private final BlockingQueue\u0026lt;T\u0026gt; queue; private final int capacity; private final MeterRegistry registry; public MonitoredQueue(int capacity, MeterRegistry registry) { this.queue = new ArrayBlockingQueue\u0026lt;\u0026gt;(capacity); this.capacity = capacity; this.registry = registry; // Monitor queue depth registry.gauge(\u0026#34;queue.size\u0026#34;, queue, BlockingQueue::size); registry.gauge(\u0026#34;queue.utilization\u0026#34;, queue, q -\u0026gt; (double) q.size() / capacity); } public void put(T item) throws InterruptedException { long startTime = System.nanoTime(); queue.put(item); long duration = System.nanoTime() - startTime; registry.timer(\u0026#34;queue.put.time\u0026#34;).record(duration, TimeUnit.NANOSECONDS); // Alert if blocking for too long if (duration \u0026gt; TimeUnit.SECONDS.toNanos(1)) { logger.warn(\u0026#34;Producer blocked for {}ms\u0026#34;, TimeUnit.NANOSECONDS.toMillis(duration)); } } } Part 2 Summary: Core Synchronization Patterns We\u0026rsquo;ve covered 4 fundamental synchronization tools:\nTool Best For Key Limitation CyclicBarrier Fixed multi-phase work Can\u0026rsquo;t change party count Semaphore Resource limiting Must remember to release ThreadFactory Thread customization Doesn\u0026rsquo;t prevent leaks BlockingQueue Producer-consumer Queue can grow unbounded Key Takeaways:\nCyclicBarrier resets after each phase—use for iterative multi-phase work Semaphore limits resource access—always release in try-finally ThreadFactory names threads—makes debugging at 2 AM possible BlockingQueue separates producers from consumers—always use bounded queues Coming in Part 3 (Day 97):\nDelayQueue - Time-delayed execution with exponential backoff ReentrantLock - Fine-grained locking with timeouts Phaser - Dynamic multi-phase coordination Plus: Debugging techniques, monitoring patterns, and production readiness checklist Previous: Day 95 - Java Concurrency Toolkit Part 1: Foundation \u0026amp; Execution Patterns\nNext: Day 97 - Java Concurrency Toolkit Part 3: Advanced Patterns \u0026amp; Production Readiness\n","permalink":"https://mohibul.pages.dev/posts/posts/java/100daysofjava/day96/","summary":"Part 2: Core Synchronization Patterns - Deep dive into 4 essential coordination tools with real-world patterns, performance characteristics, and common pitfalls","title":"Day 96: Java Concurrency Toolkit Part 2 - Core Synchronization Patterns"},{"content":" Series Navigation: Part 1 (You are here) • Part 2: Core Synchronization • Part 3: Advanced Patterns\n\u0026ldquo;Concurrency is not parallelism. Concurrency is about dealing with lots of things at once. Parallelism is about doing lots of things at once.\u0026rdquo; - Rob Pike\nI\u0026rsquo;ve worked with Executor, ScheduledExecutorService, Future, CompletableFuture, CountDownLatch, ReentrantLock in production, and they\u0026rsquo;ve helped me develop various multi-threaded features. But I realized I was barely scratching the surface of what java.util.concurrent offers.\nPrimary References:\nGoetz, B., et al. (2006). Java Concurrency in Practice. Addison-Wesley. Oracle. (2023). Java SE API Specification: java.util.concurrent package Lea, D. (2000). \u0026ldquo;A Java Fork/Join Framework\u0026rdquo;, Science of Computer Programming, 37(1-3). This is Part 1 of 3 where we\u0026rsquo;ll explore the foundation and execution patterns: the core building blocks that power modern concurrent Java applications.\nSeries Overview This is a 3-part series on Java Concurrency:\nPart 1 (Today - Day 95): Foundation \u0026amp; Execution\nExecutor ExecutorService ScheduledExecutorService Future CompletableFuture CountDownLatch Part 2 (Day 96): Core Synchronization Patterns\nCyclicBarrier Semaphore ThreadFactory BlockingQueue Part 3 (Day 97): Advanced Patterns \u0026amp; Production Readiness\nDelayQueue ReentrantLock Phaser Debugging, Monitoring, Production Patterns Quick Reference: Part 1 Tools at a Glance Tool TL;DR Key Gotcha Executor Simplest interface - just execute(Runnable). No lifecycle, no results. No way to retrieve results or manage lifecycle ExecutorService Managed thread pool with lifecycle control. Returns Future for results. Must call shutdown() - thread pools leak otherwise ScheduledExecutorService Schedules tasks for future or periodic execution. Exceptions cancel all future executions - wrap in try-catch! Future Represents eventual completion. Always use get(timeout) - never without. Blocking API - blocks calling thread until complete CompletableFuture Non-blocking async composition (Java\u0026rsquo;s Promises). Chain, combine, handle errors. Complex API - use thenCompose not thenApply(join()) CountDownLatch One-shot countdown. Wait at await() until countDown() reaches zero. Can\u0026rsquo;t reset. Always countDown in finally to avoid deadlock When NOT to Use Concurrency Before we dive in, let\u0026rsquo;s talk about when concurrency is overkill:\nDon\u0026rsquo;t use concurrency when:\nYour dataset fits in memory and processes in \u0026lt; 100ms sequentially The overhead of thread coordination exceeds the benefit Your task is inherently sequential (step B depends on step A\u0026rsquo;s result) You\u0026rsquo;re processing \u0026lt; 1000 items and each takes microseconds Debugging complexity isn\u0026rsquo;t worth the performance gain Do use concurrency when:\nYou have independent tasks that can run in parallel You\u0026rsquo;re doing I/O (network, disk, database) where threads spend time waiting You need to keep your application responsive while background work happens You\u0026rsquo;re processing large datasets where parallel processing yields measurable gains You need to handle multiple concurrent requests (web servers, message processors) Rule of thumb: Measure first. Add concurrency only when sequential processing is proven to be a bottleneck.\nThe Cost Model: Concurrency introduces coordination overhead. By Amdahl\u0026rsquo;s Law, speedup is limited by the sequential fraction of your program (Goetz et al., 2006, Ch. 11). If synchronization overhead exceeds the parallelizable benefit, you\u0026rsquo;ll make things slower. Always profile before parallelizing.\nPerformance Characteristics Reference Tool CPU Overhead Memory Characteristics Behavioral Notes Best For Executor Low Minimal; delegates to provided threads Submits tasks without lifecycle management Fire-and-forget tasks ExecutorService Low–Medium Thread-pool backed; scales with pool size and queue bounds Manages worker threads, queues, and shutdown Managed task execution ScheduledExecutorService Medium Thread-pool plus heap-based delay queues (priority queue) Time-based and periodic task execution Delayed and recurring tasks Future Very Low Single object reference (~40-60 bytes) per task Represents eventual completion; blocking get() Simple result retrieval CompletableFuture Low–Medium Base object + callback chain (~200-400 bytes per composition) Non-blocking composition, async pipelines Async workflow chains CountDownLatch Very Low ~32 bytes via AbstractQueuedSynchronizer One-shot coordination of thread arrival Release-on-countdown scenarios Memory characteristics based on HotSpot JVM defaults. Thread stack size typically 1MB per thread (configurable via -Xss). See Oracle JVM documentation for platform-specific values.\nThe Foundation: Understanding What We\u0026rsquo;re Building Before we dive into the tools, let\u0026rsquo;s get one thing straight. Concurrency in Java isn\u0026rsquo;t about making everything parallel. It\u0026rsquo;s about structuring your code to handle multiple concerns efficiently. Sometimes that means parallel execution. Sometimes it means coordinating sequential tasks across threads. Sometimes it\u0026rsquo;s about protecting shared state from chaos.\nThink of it like a busy restaurant kitchen:\nA head chef coordinating tasks (Executor) Managed cooking stations (ExecutorService) Timer-based prep work (ScheduledExecutorService) Order tickets (Future) Complex recipe flows (CompletableFuture) Waiting for all dishes before serving (CountDownLatch) 1. Executor: The Fire-and-Forget Pattern TL;DR: Simplest interface - just execute(Runnable). No lifecycle, no results. Perfect for async side effects like logging or metrics where you don\u0026rsquo;t need to wait for completion.\nThe Problem: Your request thread is too precious to waste on side tasks like logging or metrics.\nThis is one pattern I\u0026rsquo;ve used extensively. When you\u0026rsquo;re processing API requests, offloading non-critical work like logging enrichment to background threads can significantly improve response times.\nThe Solution: Offload non-critical work to background threads.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 import java.util.concurrent.Executor; import java.time.Instant; import java.util.concurrent.Executors; public class AsyncLogger { private final Executor logExecutor = Executors.newSingleThreadExecutor(r -\u0026gt; { Thread t = new Thread(r); t.setName(\u0026#34;async-logger\u0026#34;); t.setDaemon(true); // Won\u0026#39;t prevent JVM shutdown return t; }); public void logRequestAsync(String userId, String endpoint, int statusCode) { // Request thread returns immediately logExecutor.execute(() -\u0026gt; { // Background thread does the heavy lifting String enrichedLog = enrichLog(userId, endpoint, statusCode); writeToElasticsearch(enrichedLog); }); } private String enrichLog(String userId, String endpoint, int statusCode) { // Fetch user details, geo-location, etc. return String.format(\u0026#34;[%s] User: %s | Endpoint: %s | Status: %d\u0026#34;, Instant.now(), userId, endpoint, statusCode); } private void writeToElasticsearch(String log) { System.out.println(\u0026#34;Writing to ES: \u0026#34; + log); } } Key Insight: Executor is the simplest interface—just execute(Runnable). No lifecycle management, no result handling. Perfect for fire-and-forget side effects.\nTrade-offs and Limitations Pros:\nMinimal overhead Simple API Request thread returns immediately Cons:\nNo result retrieval mechanism No lifecycle management (can\u0026rsquo;t shut down gracefully) No built-in error handling Can\u0026rsquo;t limit concurrent task count When it goes wrong:\n1 2 3 4 5 6 7 8 // DANGER: Unbounded task submission Executor executor = Executors.newCachedThreadPool(); for (int i = 0; i \u0026lt; 1_000_000; i++) { executor.execute(() -\u0026gt; { // If tasks take longer than submission rate, // you\u0026#39;ll create 1 million threads = OOM! }); } Reference: Oracle Java SE API - Executors.newCachedThreadPool(): \u0026ldquo;Creates a thread pool that creates new threads as needed, but will reuse previously constructed threads when they are available.\u0026rdquo; - No upper bound documented.\nCommon Mistakes:\nUsing daemon threads for critical work (logs might be lost on JVM shutdown) - Reference: Java Language Specification §12.8: The JVM exits when all non-daemon threads terminate. No backpressure mechanism (can overwhelm the executor) Losing exceptions (they\u0026rsquo;re swallowed unless you add explicit handling) 2. ExecutorService: When You Need Control TL;DR: Managed thread pool with lifecycle control. Returns Future for results. Use invokeAll() for batch processing. Always shutdown: shutdown() → awaitTermination() → shutdownNow(). Size pool based on workload (CPU vs I/O).\nThe Problem: Processing a large product catalog where each item needs enrichment from multiple services—pricing, inventory, and reviews. Sequential processing is too slow.\nThis is a classic scenario where parallel processing can provide speedup—if I/O wait time dominates compute time. Concurrency benefit depends on latency distribution and CPU contention (Goetz et al., 2006, Ch. 8). Instead of waiting for each item to complete before starting the next, we can process multiple items concurrently.\nThe Solution: Parallel processing with controlled thread pool and result collection.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 import java.util.concurrent.*; import java.util.List; import java.util.ArrayList; import java.util.stream.Collectors; public class ProductEnrichmentService { private final ExecutorService pool; public ProductEnrichmentService(int threadCount) { this.pool = Executors.newFixedThreadPool(threadCount); } public List\u0026lt;Product\u0026gt; enrichProducts(List\u0026lt;Product\u0026gt; products) throws InterruptedException { System.out.println(\u0026#34;Enriching \u0026#34; + products.size() + \u0026#34; products using \u0026#34; + Runtime.getRuntime().availableProcessors() + \u0026#34; threads\u0026#34;); // Convert products to Callable tasks List\u0026lt;Callable\u0026lt;Product\u0026gt;\u0026gt; tasks = products.stream() .map(product -\u0026gt; (Callable\u0026lt;Product\u0026gt;) () -\u0026gt; enrichSingleProduct(product)) .collect(Collectors.toList()); long startTime = System.currentTimeMillis(); // Execute all tasks and wait for completion List\u0026lt;Future\u0026lt;Product\u0026gt;\u0026gt; futures = pool.invokeAll(tasks); // Collect results List\u0026lt;Product\u0026gt; enriched = new ArrayList\u0026lt;\u0026gt;(); for (Future\u0026lt;Product\u0026gt; future : futures) { try { enriched.add(future.get()); } catch (ExecutionException e) { System.err.println(\u0026#34;Failed to enrich product: \u0026#34; + e.getCause().getMessage()); } } long duration = System.currentTimeMillis() - startTime; System.out.println(\u0026#34;Enrichment completed in \u0026#34; + duration + \u0026#34;ms\u0026#34;); return enriched; } private Product enrichSingleProduct(Product product) { // Simulate calling multiple services String pricing = fetchPricing(product.id()); String inventory = fetchInventory(product.id()); String reviews = fetchReviews(product.id()); return new Product( product.id(), product.name(), pricing, inventory, reviews ); } private String fetchPricing(String id) { sleep(50); // Simulate API call return \u0026#34;$99.99\u0026#34;; } private String fetchInventory(String id) { sleep(30); return \u0026#34;In Stock\u0026#34;; } private String fetchReviews(String id) { sleep(40); return \u0026#34;4.5 stars\u0026#34;; } public void shutdown() { pool.shutdown(); try { if (!pool.awaitTermination(60, TimeUnit.SECONDS)) { pool.shutdownNow(); } } catch (InterruptedException e) { pool.shutdownNow(); } } private void sleep(long ms) { try { Thread.sleep(ms); } catch (InterruptedException e) {} } } record Product(String id, String name, String pricing, String inventory, String reviews) {} Trade-offs and Limitations Pros:\nLifecycle management (shutdown, awaitTermination) Result collection via Future Controlled parallelism Can batch submit tasks Cons:\nFixed thread pool size (need to tune) Blocking result retrieval No built-in composition Manual error handling Sizing Your Thread Pool:\n1 2 3 4 5 6 7 8 9 10 // For CPU-bound tasks: int cpuBoundPoolSize = Runtime.getRuntime().availableProcessors(); // For I/O-bound tasks (rule of thumb): int ioBoundPoolSize = Runtime.getRuntime().availableProcessors() * 2; // More precise I/O formula (derived from Little\u0026#39;s Law): // ThreadCount = NumCores * (1 + WaitTime/ComputeTime) // Example: 8 cores, 90% wait time, 10% compute // = 8 * (1 + 0.9/0.1) = 8 * 10 = 80 threads Reference: Goetz et al., 2006, Chapter 8 (\u0026ldquo;Applying Thread Pools\u0026rdquo;): Thread pool sizing using Little\u0026rsquo;s Law and utilization analysis.\nCommon Mistakes:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 // MISTAKE 1: Forgetting to shutdown public void process() { ExecutorService pool = Executors.newFixedThreadPool(10); pool.submit(() -\u0026gt; doWork()); // Pool never shuts down = thread leak! } // CORRECT: Always shutdown public void process() { ExecutorService pool = Executors.newFixedThreadPool(10); try { pool.submit(() -\u0026gt; doWork()).get(); } finally { pool.shutdown(); pool.awaitTermination(60, TimeUnit.SECONDS); } } // MISTAKE 2: Wrong pool type ExecutorService pool = Executors.newCachedThreadPool(); // For I/O: Can create unlimited threads = OOM // MISTAKE 3: Ignoring RejectedExecutionException pool.shutdown(); pool.submit(() -\u0026gt; doWork()); // Throws RejectedExecutionException Modern Java Note: Executors factory methods are convenient but lack configurability. For production systems, consider direct ThreadPoolExecutor construction with explicit queue bounds and rejection policies. Java 21+ introduces Structured Concurrency (JEP 453) and Scoped Values (JEP 446) for safer concurrent patterns.\nReference: OpenJDK JEP 453: Structured Concurrency (Preview) - https://openjdk.org/jeps/453\n3. ScheduledExecutorService: Time-Based Automation TL;DR: Schedules tasks for future or periodic execution. Use scheduleAtFixedRate() for fixed intervals, scheduleWithFixedDelay() for guaranteed gaps. Wrap tasks in try-catch - exceptions cancel future executions!\nThe Problem: OAuth tokens expire every hour. If we don\u0026rsquo;t refresh them, API calls start failing. Can\u0026rsquo;t rely on manual intervention.\nThis is another pattern I\u0026rsquo;ve implemented in production. Scheduled tasks are perfect for periodic maintenance work like token refresh, health checks, or cache cleanup.\nThe Solution: Automated token refresh with scheduled tasks.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 import java.util.concurrent.*; import java.time.Instant; import java.util.concurrent.atomic.AtomicReference; public class TokenRefreshService { private final ScheduledExecutorService scheduler; private final AtomicReference\u0026lt;String\u0026gt; currentToken; public TokenRefreshService() { this.scheduler = Executors.newSingleThreadScheduledExecutor(r -\u0026gt; { Thread t = new Thread(r); t.setName(\u0026#34;token-refresher\u0026#34;); t.setDaemon(false); // Important: ensure refresh completes before shutdown return t; }); this.currentToken = new AtomicReference\u0026lt;\u0026gt;(\u0026#34;\u0026#34;); } public void startRefreshCycle() { System.out.println(\u0026#34;Starting token refresh cycle at \u0026#34; + Instant.now()); // Initial refresh refreshToken(); // Schedule periodic refresh every 50 minutes (tokens expire in 60) scheduler.scheduleAtFixedRate( this::refreshToken, 50, // initial delay 50, // period TimeUnit.MINUTES ); System.out.println(\u0026#34;Token will refresh every 50 minutes\u0026#34;); } private void refreshToken() { try { System.out.println(\u0026#34;[\u0026#34; + Instant.now() + \u0026#34;] Refreshing token...\u0026#34;); // Simulate OAuth flow String newToken = callAuthServer(); currentToken.set(newToken); System.out.println(\u0026#34;Token refreshed successfully\u0026#34;); } catch (Exception e) { System.err.println(\u0026#34;Token refresh failed: \u0026#34; + e.getMessage()); // In production: alert ops team, retry with exponential backoff } } private String callAuthServer() { // Simulate network delay try { Thread.sleep(200); } catch (InterruptedException e) {} return \u0026#34;token_\u0026#34; + System.currentTimeMillis(); } public String getCurrentToken() { return currentToken.get(); } public void shutdown() { scheduler.shutdown(); } } scheduleAtFixedRate vs scheduleWithFixedDelay 1 2 3 4 5 6 7 // scheduleAtFixedRate: Next task starts at T + period (can overlap!) scheduler.scheduleAtFixedRate(task, 0, 1, TimeUnit.SECONDS); // If task takes 2 seconds, tasks will queue up! // scheduleWithFixedDelay: Next task starts after (task completion + delay) scheduler.scheduleWithFixedDelay(task, 0, 1, TimeUnit.SECONDS); // Guarantees 1 second gap between task completion and next start Trade-offs and Limitations Pros:\nBuilt-in timing mechanism Periodic execution One-shot delayed execution Multiple scheduled tasks on same scheduler Cons:\nIf task takes longer than period, tasks queue up (scheduleAtFixedRate) Exceptions cancel future executions - Reference: Oracle Java SE API - ScheduledExecutorService: \u0026ldquo;If any execution of the task encounters an exception, subsequent executions are suppressed.\u0026rdquo; No built-in retry logic Clock drift not handled Common Mistakes:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 // MISTAKE: Long-running task with scheduleAtFixedRate scheduler.scheduleAtFixedRate(() -\u0026gt; { Thread.sleep(5000); // 5 seconds }, 0, 1, TimeUnit.SECONDS); // Every 1 second // Tasks pile up! Queue grows indefinitely // CORRECT: Use scheduleWithFixedDelay for long tasks scheduler.scheduleWithFixedDelay(() -\u0026gt; { Thread.sleep(5000); }, 0, 1, TimeUnit.SECONDS); // Next task starts 1 second AFTER previous completes // MISTAKE: Not handling exceptions scheduler.scheduleAtFixedRate(() -\u0026gt; { if (Math.random() \u0026gt; 0.5) { throw new RuntimeException(\u0026#34;Boom!\u0026#34;); } doWork(); }, 0, 1, TimeUnit.SECONDS); // First exception silently cancels ALL future executions! // CORRECT: Wrap in try-catch scheduler.scheduleAtFixedRate(() -\u0026gt; { try { if (Math.random() \u0026gt; 0.5) { throw new RuntimeException(\u0026#34;Boom!\u0026#34;); } doWork(); } catch (Exception e) { System.err.println(\u0026#34;Task failed: \u0026#34; + e.getMessage()); // Task continues running } }, 0, 1, TimeUnit.SECONDS); 4. Future: Timeouts Save Lives TL;DR: Represents eventual completion of async task. Always use get(timeout) - never get() without timeout. Can cancel running tasks (if they respect interruption). Blocking API - use CompletableFuture for non-blocking.\nThe Problem: Calling external services that might hang or respond slowly. Without timeouts, your application threads get stuck waiting indefinitely.\nTimeout enforcement is critical for resilient systems. Whether it\u0026rsquo;s a payment gateway, external API, or database query, you need a way to fail fast and recover gracefully.\nThe Solution: Strict timeout enforcement with Future.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 import java.util.concurrent.*; public class PaymentGateway { private final ExecutorService pool; private final long timeoutMs; public PaymentGateway(long timeoutMs) { this.pool = Executors.newFixedThreadPool(10); this.timeoutMs = timeoutMs; } public PaymentResult processPayment(PaymentRequest request) { System.out.println(\u0026#34;Processing payment for $\u0026#34; + request.amount()); Future\u0026lt;PaymentResult\u0026gt; future = pool.submit(() -\u0026gt; { return callExternalGateway(request); }); try { // Enforce strict timeout PaymentResult result = future.get(timeoutMs, TimeUnit.MILLISECONDS); System.out.println(\u0026#34;Payment successful: \u0026#34; + result.transactionId()); return result; } catch (TimeoutException e) { // Cancel the stuck task future.cancel(true); System.err.println(\u0026#34;Payment timed out after \u0026#34; + timeoutMs + \u0026#34;ms\u0026#34;); return new PaymentResult(\u0026#34;TIMEOUT\u0026#34;, \u0026#34;Payment gateway timeout\u0026#34;); } catch (ExecutionException e) { System.err.println(\u0026#34;Payment failed: \u0026#34; + e.getCause().getMessage()); return new PaymentResult(\u0026#34;FAILED\u0026#34;, e.getCause().getMessage()); } catch (InterruptedException e) { Thread.currentThread().interrupt(); return new PaymentResult(\u0026#34;INTERRUPTED\u0026#34;, \u0026#34;Payment interrupted\u0026#34;); } } private PaymentResult callExternalGateway(PaymentRequest request) { // Simulate variable latency try { Thread.sleep(ThreadLocalRandom.current().nextInt(100, 2000)); } catch (InterruptedException e) { throw new RuntimeException(\u0026#34;Gateway call interrupted\u0026#34;); } return new PaymentResult(\u0026#34;TXN_\u0026#34; + System.currentTimeMillis(), \u0026#34;SUCCESS\u0026#34;); } public void shutdown() { pool.shutdown(); } } record PaymentRequest(String userId, double amount) {} record PaymentResult(String transactionId, String status) {} Trade-offs and Limitations Pros:\nSimple timeout enforcement Can cancel running tasks Works with any ExecutorService Exception handling built-in Cons:\nBlocking API (future.get() blocks) No composition (can\u0026rsquo;t chain Futures) Cancellation may not work (task must respect interruption) One result only, no streaming Critical Lesson: Always set timeouts on external calls. A slow dependency without timeout protection can create cascading failures throughout your system.\nCommon Mistakes:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 // MISTAKE: future.get() without timeout Future\u0026lt;String\u0026gt; future = pool.submit(() -\u0026gt; callExternalApi()); String result = future.get(); // Blocks FOREVER if API hangs! // CORRECT: Always use timeout Future\u0026lt;String\u0026gt; future = pool.submit(() -\u0026gt; callExternalApi()); String result = future.get(5, TimeUnit.SECONDS); // MISTAKE: Assuming cancel() stops the task Future\u0026lt;?\u0026gt; future = pool.submit(() -\u0026gt; { while (true) { // Ignores interruption! doWork(); } }); future.cancel(true); // Task keeps running! // CORRECT: Respect interruption Future\u0026lt;?\u0026gt; future = pool.submit(() -\u0026gt; { while (!Thread.currentThread().isInterrupted()) { doWork(); if (Thread.interrupted()) break; // Check interruption } System.out.println(\u0026#34;Task gracefully stopped\u0026#34;); }); future.cancel(true); // Task stops Reference: Goetz et al., 2006, Chapter 7 (\u0026ldquo;Cancellation and Shutdown\u0026rdquo;): Proper interruption handling patterns.\n5. CompletableFuture: Async Pipelines TL;DR: Non-blocking async composition (Java\u0026rsquo;s answer to Promises). Chain with thenApply/thenCompose, combine with allOf/anyOf, handle errors with exceptionally/handle. Powerful but complex - practice required. Always specify executor pool.\nThe Problem: You need to call multiple services in a chain, handle errors at each step, and compose results—all without blocking threads.\nCompletableFuture is Java\u0026rsquo;s way to JavaScript Promises. It\u0026rsquo;s powerful, composable, and non-blocking when used correctly.\nThe Solution: Build async pipelines with proper error handling.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 import java.util.concurrent.*; import java.time.Duration; public class OrderProcessingService { private final ExecutorService pool; public OrderProcessingService() { this.pool = Executors.newFixedThreadPool( Runtime.getRuntime().availableProcessors() * 2 ); } // Example 1: Sequential async chain public CompletableFuture\u0026lt;OrderConfirmation\u0026gt; processOrder(String orderId) { return CompletableFuture // Step 1: Validate order .supplyAsync(() -\u0026gt; validateOrder(orderId), pool) // Step 2: Check inventory (depends on validation) .thenCompose(order -\u0026gt; CompletableFuture.supplyAsync(() -\u0026gt; checkInventory(order), pool) ) // Step 3: Process payment (depends on inventory) .thenCompose(order -\u0026gt; CompletableFuture.supplyAsync(() -\u0026gt; processPayment(order), pool) ) // Step 4: Ship order (depends on payment) .thenApply(order -\u0026gt; shipOrder(order)) // Handle any errors in the chain .exceptionally(ex -\u0026gt; { System.err.println(\u0026#34;Order processing failed: \u0026#34; + ex.getMessage()); return new OrderConfirmation(\u0026#34;FAILED\u0026#34;, ex.getMessage()); }) // Add timeout .orTimeout(10, TimeUnit.SECONDS) // Fallback if timeout .exceptionally(ex -\u0026gt; { if (ex instanceof TimeoutException) { return new OrderConfirmation(\u0026#34;TIMEOUT\u0026#34;, \u0026#34;Processing timed out\u0026#34;); } return new OrderConfirmation(\u0026#34;ERROR\u0026#34;, ex.getMessage()); }); } // Example 2: Parallel async calls public CompletableFuture\u0026lt;ProductDetails\u0026gt; getProductDetails(String productId) { // Make three independent API calls in parallel CompletableFuture\u0026lt;String\u0026gt; pricingFuture = CompletableFuture.supplyAsync(() -\u0026gt; fetchPricing(productId), pool); CompletableFuture\u0026lt;String\u0026gt; inventoryFuture = CompletableFuture.supplyAsync(() -\u0026gt; fetchInventory(productId), pool); CompletableFuture\u0026lt;String\u0026gt; reviewsFuture = CompletableFuture.supplyAsync(() -\u0026gt; fetchReviews(productId), pool); // Combine all results return CompletableFuture.allOf(pricingFuture, inventoryFuture, reviewsFuture) .thenApply(v -\u0026gt; new ProductDetails( productId, pricingFuture.join(), // Safe: already completed inventoryFuture.join(), reviewsFuture.join() )); } // Example 3: Race multiple services (use fastest) public CompletableFuture\u0026lt;String\u0026gt; getQuote(String symbol) { CompletableFuture\u0026lt;String\u0026gt; exchange1 = CompletableFuture.supplyAsync(() -\u0026gt; callExchange1(symbol), pool); CompletableFuture\u0026lt;String\u0026gt; exchange2 = CompletableFuture.supplyAsync(() -\u0026gt; callExchange2(symbol), pool); CompletableFuture\u0026lt;String\u0026gt; exchange3 = CompletableFuture.supplyAsync(() -\u0026gt; callExchange3(symbol), pool); // Return whichever completes first return CompletableFuture.anyOf(exchange1, exchange2, exchange3) .thenApply(result -\u0026gt; (String) result); } // Example 4: Complex error handling public CompletableFuture\u0026lt;String\u0026gt; robustServiceCall(String request) { return CompletableFuture .supplyAsync(() -\u0026gt; callUnreliableService(request), pool) // Handle specific exceptions differently .handle((result, ex) -\u0026gt; { if (ex != null) { if (ex.getCause() instanceof TimeoutException) { return \u0026#34;TIMEOUT_FALLBACK\u0026#34;; } else if (ex.getCause() instanceof ConnectException) { return \u0026#34;OFFLINE_FALLBACK\u0026#34;; } else { throw new CompletionException(ex); } } return result; }) // Retry logic with delay .exceptionallyCompose(ex -\u0026gt; { System.out.println(\u0026#34;Retrying after failure...\u0026#34;); return CompletableFuture .delayedExecutor(1, TimeUnit.SECONDS, pool) .execute(() -\u0026gt; {}) .thenCompose(v -\u0026gt; CompletableFuture.supplyAsync( () -\u0026gt; callUnreliableService(request), pool ) ); }); } // Helper methods (simulated) private Order validateOrder(String orderId) { sleep(100); return new Order(orderId, \u0026#34;VALIDATED\u0026#34;); } private Order checkInventory(Order order) { sleep(150); return new Order(order.id(), \u0026#34;INVENTORY_OK\u0026#34;); } private Order processPayment(Order order) { sleep(200); return new Order(order.id(), \u0026#34;PAID\u0026#34;); } private OrderConfirmation shipOrder(Order order) { sleep(100); return new OrderConfirmation(\u0026#34;SUCCESS\u0026#34;, \u0026#34;Order shipped\u0026#34;); } private String fetchPricing(String id) { sleep(50); return \u0026#34;$99.99\u0026#34;; } private String fetchInventory(String id) { sleep(75); return \u0026#34;In Stock\u0026#34;; } private String fetchReviews(String id) { sleep(60); return \u0026#34;4.5 stars\u0026#34;; } private String callExchange1(String symbol) { sleep(100); return \u0026#34;Exchange1: $150.25\u0026#34;; } private String callExchange2(String symbol) { sleep(80); return \u0026#34;Exchange2: $150.30\u0026#34;; } private String callExchange3(String symbol) { sleep(120); return \u0026#34;Exchange3: $150.20\u0026#34;; } private String callUnreliableService(String request) { sleep(100); if (Math.random() \u0026gt; 0.7) { throw new RuntimeException(\u0026#34;Service temporarily unavailable\u0026#34;); } return \u0026#34;SUCCESS: \u0026#34; + request; } private void sleep(long ms) { try { Thread.sleep(ms); } catch (InterruptedException e) {} } public void shutdown() { pool.shutdown(); } } record Order(String id, String status) {} record OrderConfirmation(String status, String message) {} record ProductDetails(String id, String pricing, String inventory, String reviews) {} CompletableFuture Key Methods: Method Use Case Returns supplyAsync() Start async computation with result CompletableFuture runAsync() Start async computation without result CompletableFuture thenApply() Transform result (sync) CompletableFuture thenApplyAsync() Transform result (async) CompletableFuture thenCompose() Chain dependent futures (flatMap) CompletableFuture thenCombine() Combine two independent futures CompletableFuture allOf() Wait for all to complete CompletableFuture anyOf() Wait for first to complete CompletableFuture exceptionally() Handle error, return fallback CompletableFuture handle() Handle both success and error CompletableFuture orTimeout() Add timeout CompletableFuture completeOnTimeout() Provide default on timeout CompletableFuture Trade-offs and Limitations Pros:\nNon-blocking composition Rich API for chaining Built-in error handling Timeout support (Java 9+) Can combine multiple futures Cons:\nComplex API with many methods Easy to create memory leaks (uncompleted futures) Exception handling can be tricky Default pool (ForkJoinPool.commonPool()) might not be suitable Common Mistakes:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 // MISTAKE 1: Using join() instead of thenCompose() CompletableFuture\u0026lt;User\u0026gt; userFuture = getUserAsync(userId); CompletableFuture\u0026lt;Orders\u0026gt; ordersFuture = getOrdersAsync(userFuture.join().getId()); // BLOCKS! // CORRECT: Chain with thenCompose CompletableFuture\u0026lt;Orders\u0026gt; ordersFuture = getUserAsync(userId) .thenCompose(user -\u0026gt; getOrdersAsync(user.getId())); // Non-blocking // MISTAKE 2: Not handling errors CompletableFuture\u0026lt;String\u0026gt; future = CompletableFuture .supplyAsync(() -\u0026gt; { throw new RuntimeException(\u0026#34;Boom!\u0026#34;); }); // Exception is swallowed until you call get() or join() // CORRECT: Handle errors CompletableFuture\u0026lt;String\u0026gt; future = CompletableFuture .supplyAsync(() -\u0026gt; { throw new RuntimeException(\u0026#34;Boom!\u0026#34;); }) .exceptionally(ex -\u0026gt; { System.err.println(\u0026#34;Error: \u0026#34; + ex.getMessage()); return \u0026#34;FALLBACK\u0026#34;; }); // MISTAKE 4: Blocking in async callbacks CompletableFuture.supplyAsync(() -\u0026gt; step1()) .thenApply(result -\u0026gt; { return expensiveBlockingCall(); // Blocks thread pool! }); // CORRECT: Use thenApplyAsync for blocking ops CompletableFuture.supplyAsync(() -\u0026gt; step1()) .thenApplyAsync(result -\u0026gt; { return expensiveBlockingCall(); // Runs on separate thread }, customPool); 6. CountDownLatch: Startup Coordination TL;DR: One-shot countdown mechanism. Threads wait at await() until count reaches zero via countDown() calls. Can\u0026rsquo;t reset (use CyclicBarrier for reusable). Always countDown in finally block to avoid deadlock.\nThe Problem: Starting a microservice that depends on multiple other services being healthy. If you start serving traffic before dependencies are ready, requests fail.\nThis pattern is essential for graceful application startup, especially in microservice architectures where you need to coordinate multiple async initialization tasks.\nThe Solution: Wait for all health checks before accepting traffic.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 import java.util.concurrent.*; import java.util.List; public class MicroserviceBootstrap { public void startApplication() throws InterruptedException { System.out.println(\u0026#34;Starting application bootstrap...\u0026#34;); List\u0026lt;String\u0026gt; dependencies = List.of( \u0026#34;user-service\u0026#34;, \u0026#34;payment-service\u0026#34;, \u0026#34;inventory-service\u0026#34; ); CountDownLatch healthCheckLatch = new CountDownLatch(dependencies.size()); ExecutorService pool = Executors.newCachedThreadPool(); // Launch parallel health checks for (String service : dependencies) { pool.submit(() -\u0026gt; { boolean healthy = checkServiceHealth(service); if (healthy) { System.out.println(service + \u0026#34; is healthy\u0026#34;); healthCheckLatch.countDown(); } else { System.err.println(service + \u0026#34; health check failed\u0026#34;); // In production: implement retry logic } }); } System.out.println(\u0026#34;Waiting for all health checks to complete...\u0026#34;); // Block until all services are healthy or timeout boolean allHealthy = healthCheckLatch.await(30, TimeUnit.SECONDS); if (allHealthy) { System.out.println(\u0026#34;All dependencies healthy. Starting HTTP server...\u0026#34;); startHttpServer(); } else { System.err.println(\u0026#34;Health checks timed out. Application startup failed.\u0026#34;); System.exit(1); } pool.shutdown(); } private boolean checkServiceHealth(String service) { try { // Simulate HTTP health check Thread.sleep(ThreadLocalRandom.current().nextInt(500, 2000)); return true; // Assume success for demo } catch (InterruptedException e) { return false; } } private void startHttpServer() { System.out.println(\u0026#34;HTTP server started on port 8080\u0026#34;); } } Visualizing CountDownLatch Trade-offs and Limitations Pros:\nSimple API (just countDown and await) Very low overhead (~32 bytes) Can have multiple waiters Can timeout Cons:\nOne-shot only (can\u0026rsquo;t reset) Count must be known at creation No error propagation If countDown() is missed, waiters block forever Why CountDownLatch? It\u0026rsquo;s a one-shot synchronization primitive. Once the count reaches zero, it\u0026rsquo;s done. Perfect for startup coordination where you only need to wait once.\nCommon Mistakes:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 // MISTAKE: Wrong count CountDownLatch latch = new CountDownLatch(5); for (int i = 0; i \u0026lt; 3; i++) { // Only 3 threads! pool.submit(() -\u0026gt; { doWork(); latch.countDown(); }); } latch.await(); // Blocks forever! Count never reaches 0 // CORRECT: Count matches threads int threadCount = 3; CountDownLatch latch = new CountDownLatch(threadCount); for (int i = 0; i \u0026lt; threadCount; i++) { pool.submit(() -\u0026gt; { doWork(); latch.countDown(); }); } // MISTAKE: Forgetting countDown on error pool.submit(() -\u0026gt; { if (checkHealth()) { latch.countDown(); } // If check fails, count is never decremented! }); // CORRECT: Always countDown (use finally) pool.submit(() -\u0026gt; { try { checkHealth(); } finally { latch.countDown(); // Always happens } }); Common Anti-Patterns Across All Tools 1. Thread Pool Sizing 1 2 3 4 5 6 // ANTI-PATTERN: Magic numbers ExecutorService pool = Executors.newFixedThreadPool(42); // CORRECT: Calculate based on workload int poolSize = Runtime.getRuntime().availableProcessors() * (isCpuBound ? 1 : 2); 2. Ignoring Shutdown 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 // ANTI-PATTERN: No shutdown public void process() { ExecutorService pool = Executors.newFixedThreadPool(10); pool.submit(() -\u0026gt; work()); } // Pool leaks! // CORRECT: Proper shutdown public void process() { ExecutorService pool = Executors.newFixedThreadPool(10); try { pool.submit(() -\u0026gt; work()).get(); } finally { pool.shutdown(); pool.awaitTermination(60, TimeUnit.SECONDS); } } 3. Daemon Threads for Critical Work 1 2 3 4 5 6 7 8 9 10 11 12 13 // ❌ ANTI-PATTERN: Daemon thread for DB writes ExecutorService pool = Executors.newSingleThreadExecutor(r -\u0026gt; { Thread t = new Thread(r); t.setDaemon(true); // JVM can exit before DB write completes! return t; }); // ✅ CORRECT: Non-daemon for critical work ExecutorService pool = Executors.newSingleThreadExecutor(r -\u0026gt; { Thread t = new Thread(r); t.setDaemon(false); // JVM waits for completion return t; }); Debugging Concurrent Code Essential Tools:\n1. Thread Dumps with jstack 1 2 3 4 5 6 7 8 9 10 # Get process ID jps # Generate thread dump jstack \u0026lt;pid\u0026gt; \u0026gt; threaddump.txt # Look for: # - \u0026#34;BLOCKED\u0026#34; threads waiting on locks # - \u0026#34;WAITING\u0026#34; threads in park() # - Thread pool names (why naming matters!) Reference: Oracle JDK Tools - jstack: \u0026ldquo;Prints Java thread stack traces for a Java process.\u0026rdquo;\n2. ThreadPoolExecutor Hooks 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 public class MonitoredThreadPool extends ThreadPoolExecutor { private final ThreadLocal\u0026lt;Long\u0026gt; startTime = new ThreadLocal\u0026lt;\u0026gt;(); public MonitoredThreadPool(int corePoolSize, int maxPoolSize) { super(corePoolSize, maxPoolSize, 60L, TimeUnit.SECONDS, new LinkedBlockingQueue\u0026lt;\u0026gt;()); } @Override protected void beforeExecute(Thread t, Runnable r) { super.beforeExecute(t, r); startTime.set(System.nanoTime()); System.out.println(\u0026#34;Task starting on thread: \u0026#34; + t.getName()); } @Override protected void afterExecute(Runnable r, Throwable t) { try { long elapsed = System.nanoTime() - startTime.get(); System.out.println(\u0026#34;Task completed in \u0026#34; + elapsed / 1_000_000 + \u0026#34;ms\u0026#34;); if (t != null) { System.err.println(\u0026#34;Task failed: \u0026#34; + t.getMessage()); } } finally { super.afterExecute(r, t); } } @Override protected void terminated() { System.out.println(\u0026#34;ThreadPool terminated. Completed tasks: \u0026#34; + getCompletedTaskCount()); } } Reference: Oracle Java SE API - ThreadPoolExecutor: Hook methods beforeExecute, afterExecute, and terminated for monitoring and debugging.\n3. Detecting Thread Pool Saturation 1 2 3 4 5 6 7 8 9 10 11 ThreadPoolExecutor pool = (ThreadPoolExecutor) Executors.newFixedThreadPool(10); // Monitor pool health int activeCount = pool.getActiveCount(); int queueSize = pool.getQueue().size(); long completedTasks = pool.getCompletedTaskCount(); if (queueSize \u0026gt; 1000) { System.err.println(\u0026#34;WARNING: Thread pool queue saturated!\u0026#34;); // Alert ops team, scale horizontally, or apply backpressure } Reference: Goetz et al., 2006, Chapter 8 (\u0026ldquo;Applying Thread Pools\u0026rdquo;): Monitoring thread pool saturation and tuning.\n4. Common Deadlock Patterns 1 2 3 4 5 6 7 8 9 // Thread 1: locks A, waits for B synchronized(lockA) { synchronized(lockB) { /* ... */ } } // Thread 2: locks B, waits for A → DEADLOCK! synchronized(lockB) { synchronized(lockA) { /* ... */ } } Detection: jstack will report \u0026ldquo;Found one Java-level deadlock\u0026rdquo; and show the lock dependency cycle.\nPrevention: Always acquire locks in consistent order (Goetz et al., 2006, Ch. 10).\nPart 1 Summary: Foundation \u0026amp; Execution We\u0026rsquo;ve covered the foundation of Java concurrency:\nTool Best For Key Limitation Executor Fire-and-forget No result retrieval ExecutorService Managed thread pools Blocking result collection ScheduledExecutorService Periodic tasks Tasks can queue up Future Timeout enforcement Blocking API CompletableFuture Async pipelines Complex API CountDownLatch One-shot coordination Can\u0026rsquo;t reset Key Takeaways:\nAlways set timeouts - Blocking forever = production incidents Name your threads - Debugging with \u0026ldquo;pool-1-thread-17\u0026rdquo; is painful Proper shutdown is critical - shutdown() → awaitTermination() → shutdownNow() Match pool size to workload - CPU-bound vs I/O-bound need different sizing CompletableFuture is powerful - But requires practice to use correctly Coming in Part 2 (Day 96):\nCyclicBarrier - Reusable multi-phase coordination Semaphore - Resource pool management ThreadFactory - Production-ready thread management BlockingQueue - Producer-consumer patterns Coming in Part 3 (Day 97):\nDelayQueue - Time-delayed execution ReentrantLock - Fine-grained locking Phaser - Dynamic party registration Plus: Debugging techniques, monitoring patterns, and production readiness checklist Next: Day 96 - Java Concurrency Toolkit Part 2: Core Synchronization Patterns\n","permalink":"https://mohibul.pages.dev/posts/posts/java/100daysofjava/day95/","summary":"Part 1: Foundation \u0026amp; Execution - Deep dive into 6 essential concurrency tools with real-world patterns, performance characteristics, and common pitfalls","title":"Day 95: Java Concurrency Toolkit Part 1 - Foundation \u0026 Execution Patterns"},{"content":"A few days ago I attended a company knowledge sharing where one of my brilliant colleauges talked about optimizing java memory and performance tuning. Where he talked about spring boot application start up time and how we can shave off some time to load the application faster.\nThat night after thinking about it, I fell down the rabbit hole. And what I found gave me lots of ideas. Turns out, I\u0026rsquo;d been thinking about startup time all wrong.\nMy Journey Structure I started with a Jersey application startup problem, but quickly realized optimization techniques fall into three categories:\nJVM-level (Chapters 1, 4, 5) - Apply to ANY Java app Spring Boot-specific (Chapters 2, 3) - Research for future projects Framework-agnostic architecture (Chapters 6, 7) - Adaptable to Jersey I\u0026rsquo;ve marked each technique with its applicability.\nThe Uncomfortable Truth About Startup Time Here\u0026rsquo;s something nobody tells you when you\u0026rsquo;re learning Java: every second of startup time isn\u0026rsquo;t created equal.\nI used to think \u0026ldquo;JVM starts slow, deal with it.\u0026rdquo; But that\u0026rsquo;s like saying \u0026ldquo;traffic exists, deal with it\u0026rdquo; without asking why there\u0026rsquo;s traffic or where the bottleneck actually is.\nAfter digging deep (and I mean JVM logs, profilers, lots of blogs and articles), I discovered a typical breakdown that many Java developers observe:\nMajor part of time goes to classloading - The JVM reading and parsing your .class files For spring another major time spend in bean creation - Spring (or your framework) wiring everything together Another big chunk of time goes into reflection and proxies - All that runtime magic has a cost Some of the time goes into I/O - Reading configs, connecting to databases Note: These are approximate distributions based on analysis from Spring Boot team\u0026rsquo;s startup optimization research, Oracle\u0026rsquo;s JVM performance documentation, and profiling multiple applications. Your mileage will vary depending on framework, dependencies, and architecture choices.\nI was dealing with very slow startup time of the Jersey application though, my application loads and verifies a lot of stuff before starting still I think there could be improvements.Once I knew where the time was actually going, I could fight back.\nPro Tip: Before optimizing anything, jump to The Measurement Mindset to learn how to profile your specific bottlenecks. I\u0026rsquo;m presenting techniques in the order I discovered them, but you should measure first.\nChapter 1: The JVM Strikes Back (Or: How I Learned to Stop Worrying and Love AppCDS) Let\u0026rsquo;s talk about Class Data Sharing. Sounds boring, right? It\u0026rsquo;s not. It\u0026rsquo;s basically a time machine.\nHere\u0026rsquo;s the deal: Every time you start your Java app, the JVM loads core classes—String, ArrayList, all those JDK fundamentals. And every time, it does the same work. Parse the class file, verify the bytecode, prepare the internal structures.\nWhat if\u0026hellip; we could skip that?\nThe AppCDS Discovery AppCDS (Application Class Data Sharing) lets you create a snapshot of loaded classes and reuse it. Think of it like meal prepping for your JVM.\nHere\u0026rsquo;s what I did:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 # Step 1: Generate a list of classes your app loads java -XX:DumpLoadedClassList=classes.lst \\ -jar my-app.jar # Step 2: Create the shared archive java -Xshare:dump \\ -XX:SharedClassListFile=classes.lst \\ -XX:SharedArchiveFile=app-cds.jsa \\ -jar my-app.jar # Step 3: Run with the shared archive java -XX:SharedArchiveFile=app-cds.jsa \\ -Xshare:on \\ -jar my-app.jar No code changes. No architecture overhaul. Just telling the JVM to be smarter about something it was already doing.\nAPPLIES TO: Any Java 10+ application (not Spring-specific)\nChapter 2: Spring Boot\u0026rsquo;s Little Secrets I love Spring Boot. I really do. But it\u0026rsquo;s like that friend who insists on checking if you locked the door, turned off the stove, and closed the windows—even when you\u0026rsquo;re just running to the mailbox.\nNOTE: While I was exploring Jersey optimization, I researched these Spring Boot techniques. If you\u0026rsquo;re using Spring Boot, these will apply directly. For Jersey/other frameworks, focus on the JVM-level optimizations in Chapters 1, 4, and 5.\nAuto-Configuration: The Double-Edged Sword Spring Boot\u0026rsquo;s auto-configuration is brilliant for getting started fast. But in production? It\u0026rsquo;s scanning and conditionally configuring things one might never use.\nOne can have lots of auto-configurations enabled, and actually need maybe very few of them.\nHere\u0026rsquo;s the before:\n1 2 3 4 5 6 @SpringBootApplication public class MyApp { public static void main(String[] args) { SpringApplication.run(MyApp.class, args); } } Simple, right? Too simple. It was doing SO much behind the scenes.\nHere\u0026rsquo;s after I got specific:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 @SpringBootApplication(exclude = { DataSourceAutoConfiguration.class, HibernateJpaAutoConfiguration.class, JmxAutoConfiguration.class, RabbitAutoConfiguration.class, MongoAutoConfiguration.class, // ... and many others I wasn\u0026#39;t using }) public class MyApp { public static void main(String[] args) { SpringApplication app = new SpringApplication(MyApp.class); app.setBannerMode(Banner.Mode.OFF); // Sorry, Spring banner app.setRegisterShutdownHook(false); app.setLogStartupInfo(false); app.run(args); } } this will trim down some startup time when a spring boot application is starting up.\nThe Lazy Revolution Then I discovered lazy initialization. This was\u0026hellip; controversial.\n1 spring.main.lazy-initialization=true One line. Just one line, but it can have great effect.\nWhat this does is beautiful and slightly dangerous: instead of creating all your beans at startup, Spring creates them when they\u0026rsquo;re first needed.\nBut here\u0026rsquo;s the catch—and this is important—lazy initialization means errors that would\u0026rsquo;ve shown up at startup now appear later, potentially in production. So I paired it with comprehensive integration tests. You can\u0026rsquo;t just YOLO lazy initialization without safety nets.\nThe Context Indexer Nobody Talks About There\u0026rsquo;s this little-known Spring dependency that creates a compile-time index of your components:\n1 2 3 4 5 \u0026lt;dependency\u0026gt; \u0026lt;groupId\u0026gt;org.springframework\u0026lt;/groupId\u0026gt; \u0026lt;artifactId\u0026gt;spring-context-indexer\u0026lt;/artifactId\u0026gt; \u0026lt;optional\u0026gt;true\u0026lt;/optional\u0026gt; \u0026lt;/dependency\u0026gt; At compile time, it generates META-INF/spring.components. This means Spring doesn\u0026rsquo;t have to scan your entire classpath looking for @Component and @Service annotations—it already knows where they are.\nChapter 3: The Nuclear Option (GraalVM Native Image) At this point, I had to ask myself: how fast could this thing really go?\nEnter GraalVM Native Image. This is the quantum leap, the paradigm shift, the \u0026ldquo;wait, is this still Java?\u0026rdquo; moment.\nADVANCED TECHNIQUE: This requires significant build changes and may not work with all libraries. Evaluate carefully.\nWhat Actually Happens Normal Java: JVM starts → loads classes → JIT compiles hot paths → your app runs.\nNative Image: Ahead-of-time compilation → standalone binary → boom, you\u0026rsquo;re running.\nNo JVM startup. No classloading. No JIT warmup. Just your app, as a native binary, starting in milliseconds.\nHere\u0026rsquo;s what I added to my pom.xml:\n1 2 3 4 5 \u0026lt;plugin\u0026gt; \u0026lt;groupId\u0026gt;org.graalvm.buildtools\u0026lt;/groupId\u0026gt; \u0026lt;artifactId\u0026gt;native-maven-plugin\u0026lt;/artifactId\u0026gt; \u0026lt;version\u0026gt;0.9.28\u0026lt;/version\u0026gt; \u0026lt;/plugin\u0026gt; And Spring Boot config:\n1 spring.aot.enabled=true The build process became\u0026hellip; intense. It would take very long time to build.\nThe Tradeoffs (Because Nothing Is Free) Native image isn\u0026rsquo;t magic. It\u0026rsquo;s hardcore static analysis and ahead-of-time compilation. Here\u0026rsquo;s what I learned the hard way:\nReflection needs hints - If you use reflection, you need to tell GraalVM about it Build time increases - Significantly Memory at startup is fixed - No dynamic heap sizing Some libraries don\u0026rsquo;t work - Anything doing crazy bytecode manipulation might break For some use cases its Worth it. For a complex enterprise app with tons of reflection? Maybe not.\nChapter 4: The Forgotten Optimizations While we\u0026rsquo;re here, let me share some gems that don\u0026rsquo;t get enough love:\nThe Serial GC Trick For short-lived apps (CLI tools, serverless functions), use the Serial GC:\n1 -XX:+UseSerialGC G1GC is great for long-running apps, but its initialization overhead is wasted on something that runs for 30 seconds. Serial GC starts faster. Way faster.\nThe Heap Size Paradox I used to think bigger heap = better performance. For startup, it\u0026rsquo;s backwards.\nLarge heaps take longer to initialize. The JVM has to zero out that memory.\n1 2 3 4 5 # Before (my \u0026#34;bigger is better\u0026#34; phase) -Xms2g -Xmx4g # After (the \u0026#34;only what I need\u0026#34; enlightenment) -Xms64m -Xmx512m For a microservice that uses maybe 200MB at peak? This was a no-brainer.\nThe Tiered Compilation Secret Java\u0026rsquo;s JIT has multiple tiers of optimization. For startup-critical apps, you can stop at tier 1:\n1 -XX:TieredStopAtLevel=1 This means: compile with the simple JIT, skip the expensive optimizations. Your steady-state performance will be lower, but startup? Lightning fast.\nChapter 5: Build and Packaging—The Overlooked Goldmine Here\u0026rsquo;s something that I found really intriguing when I started researching about startup time reduction: how you package your app matters as much as what\u0026rsquo;s in it.\nThe Fat JAR Double Tax Ever wonder why Spring Boot JARs are called \u0026ldquo;fat JARs\u0026rdquo;? They\u0026rsquo;re JARs\u0026hellip; containing JARs. And that means your app gets hit with a double classloader overhead.\nWhen you run:\n1 java -jar my-app.jar The spring-boot-loader has to:\nUnpack nested JARs (or read them from the archive) Create a custom classloader Then load your actual classes It\u0026rsquo;s like having to open a box, to open another box, to get to your actual present.\nThe Exploded JAR Revelation Try this instead:\n1 2 3 4 5 # Extract your fat JAR unzip my-app.jar -d app/ # Run directly from the classpath java -cp \u0026#34;app/BOOT-INF/classes:app/BOOT-INF/lib/*\u0026#34; com.example.Main No nested JARs. No custom classloader. Just straightforward classpath execution.\nJLink: Your Custom JRE Builder This one\u0026rsquo;s for the hardcore optimizers. Why ship the entire JDK when you only use a fraction of it?\nJLink creates a custom Java runtime with only the modules you need:\n1 2 3 4 5 6 7 8 9 10 # First, see what modules you actually use jdeps --print-module-deps my-app.jar # Then build a custom runtime (example output) jlink --module-path $JAVA_HOME/jmods \\ --add-modules java.base,java.logging,java.sql,java.xml \\ --output custom-jre \\ --compress=2 \\ --no-header-files \\ --no-man-pages Less to load = faster startup. Plus, your Docker images shrink dramatically.\nDependency Pruning: The Spring Cleaning You\u0026rsquo;ve Been Avoiding I ran this command and I was like why do we use so many dependencies:\n1 mvn dependency:tree | grep -c \u0026#34;jar\u0026#34; found 206 dependencies.\nDid I need all of them? thats the question I asked Then I started to look which kind of dependencies one can remove?.\ndependencies you can think to remove:\nRemove test-scope dependencies leaking into runtime Excluded transitive dependencies thats not being used Replace heavy libraries with lighter alternatives if possible Example from my sample projects pom.xml:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 \u0026lt;dependency\u0026gt; \u0026lt;groupId\u0026gt;org.springframework.boot\u0026lt;/groupId\u0026gt; \u0026lt;artifactId\u0026gt;spring-boot-starter-web\u0026lt;/artifactId\u0026gt; \u0026lt;exclusions\u0026gt; \u0026lt;!-- Don\u0026#39;t need embedded Tomcat for this service --\u0026gt; \u0026lt;exclusion\u0026gt; \u0026lt;groupId\u0026gt;org.springframework.boot\u0026lt;/groupId\u0026gt; \u0026lt;artifactId\u0026gt;spring-boot-starter-tomcat\u0026lt;/artifactId\u0026gt; \u0026lt;/exclusion\u0026gt; \u0026lt;!-- Using Log4j2, don\u0026#39;t need Logback --\u0026gt; \u0026lt;exclusion\u0026gt; \u0026lt;groupId\u0026gt;org.springframework.boot\u0026lt;/groupId\u0026gt; \u0026lt;artifactId\u0026gt;spring-boot-starter-logging\u0026lt;/artifactId\u0026gt; \u0026lt;/exclusion\u0026gt; \u0026lt;/exclusions\u0026gt; \u0026lt;/dependency\u0026gt; Each JAR on the classpath is I/O overhead. Less JARs = less scanning = faster startup.\nChapter 6: Architecture Choices That Compound This is where we get strategic. Code-level tweaks are great, but architectural decisions? They multiply your wins.\nOn-Demand Initialization: The \u0026ldquo;I\u0026rsquo;ll Get to It Later\u0026rdquo; Pattern Not everything needs to be ready at startup. Database connection pools? Sure. Your recommendation engine\u0026rsquo;s ML model? Maybe not.\nHere\u0026rsquo;s what I changed:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 // Before: Everything initialized eagerly @Bean public CacheManager cacheManager() { CacheManager manager = new CaffeineCacheManager(); manager.loadAllCaches(); // This takes 2 seconds! return manager; } // After: Initialize in the background @Bean public CacheManager cacheManager() { CacheManager manager = new CaffeineCacheManager(); // Load caches asynchronously after startup CompletableFuture.runAsync(() -\u0026gt; { manager.loadAllCaches(); log.info(\u0026#34;Cache warming complete\u0026#34;); }); return manager; } My app now starts, then warms up the cache.\nParallel Initialization: Multithreading for Startup Why initialize heavy components one-by-one when you have multiple cores?\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 @Configuration public class ParallelInitConfig { @Bean public ApplicationRunner parallelInitializer( DataSource ds, CacheManager cache, MessagingClient messaging) { return args -\u0026gt; { CompletableFuture\u0026lt;Void\u0026gt; dbWarmup = CompletableFuture.runAsync(() -\u0026gt; { // Warmup database connections ds.getConnection().close(); }); CompletableFuture\u0026lt;Void\u0026gt; cacheWarmup = CompletableFuture.runAsync(() -\u0026gt; { cache.loadCriticalData(); }); CompletableFuture\u0026lt;Void\u0026gt; messagingConnect = CompletableFuture.runAsync(() -\u0026gt; { messaging.connect(); }); // Wait for all to complete CompletableFuture.allOf(dbWarmup, cacheWarmup, messagingConnect) .exceptionally(ex -\u0026gt; { logger.error(\u0026#34;Startup task failed\u0026#34;, ex); return null; }) .join(); }; } } Three slow tasks that took 4 seconds sequentially? Now they take 1.5 seconds in parallel.\nThe Static Initializer Trap I found this technique when I was researching:\n1 2 3 4 5 6 7 8 9 public class ConfigLoader { private static final Map\u0026lt;String, String\u0026gt; CONFIG = loadConfig(); private static Map\u0026lt;String, String\u0026gt; loadConfig() { // Reads a 50MB XML file // Takes 800ms return parseXml(\u0026#34;huge-config.xml\u0026#34;); } } Every time the JVM loads this class (even if we never use the config), we pay the 800ms tax.\nFixed version:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 public class ConfigLoader { private static volatile Map\u0026lt;String, String\u0026gt; CONFIG; public static Map\u0026lt;String, String\u0026gt; getConfig() { if (CONFIG == null) { synchronized (ConfigLoader.class) { if (CONFIG == null) { CONFIG = parseXml(\u0026#34;huge-config.xml\u0026#34;); } } } return CONFIG; } } Now it only loads when actually needed. If this service doesn\u0026rsquo;t use the config? We never pay the cost.\nChapter 7: Code-Level Tricks That Add Up The small stuff matters. A lot of small optimizations compound into big wins.\nThe Reflection Tax Reflection is slow. Classpath scanning for annotations is slower. You know what\u0026rsquo;s fast? Compile-time code generation.\nI replaced this:\n1 2 3 4 5 6 7 @Service public class DataMapper { public UserDTO toDTO(User user) { // Uses reflection to map fields return modelMapper.map(user, UserDTO.class); } } With MapStruct:\n1 2 3 4 5 @Mapper(componentModel = \u0026#34;spring\u0026#34;) public interface DataMapper { // Generated at compile time—zero reflection UserDTO toDTO(User user); } No runtime reflection. No classpath scanning. Just plain old method calls.\nLogging: The Silent Startup Killer Log4j2, Logback—they\u0026rsquo;re great. But their initialization? Not so much.\nI switched from this config:\n1 2 3 4 5 6 7 8 \u0026lt;!-- log4j2.xml with complex appenders and filters --\u0026gt; \u0026lt;Configuration\u0026gt; \u0026lt;Appenders\u0026gt; \u0026lt;RollingFile name=\u0026#34;file\u0026#34;\u0026gt; \u0026lt;!-- Lots of configuration --\u0026gt; \u0026lt;/RollingFile\u0026gt; \u0026lt;/Appenders\u0026gt; \u0026lt;/Configuration\u0026gt; To programmatic configuration:\n1 2 3 4 5 6 7 8 9 10 @Configuration public class LogConfig { @PostConstruct public void configureLogs() { // Configure only what we need, when we need it LoggerContext ctx = (LoggerContext) LogManager.getContext(false); Configuration config = ctx.getConfiguration(); // Minimal setup } } Or for services that don\u0026rsquo;t need fancy logging, I went nuclear:\n1 2 # Just use simple logging org.springframework.boot.autoconfigure.logging=false Precompiling Configuration Every Spring Boot startup, we were parsing this:\n1 2 3 4 5 6 7 # application.yml - 500 lines of YAML spring: datasource: # ... 100 lines jpa: # ... 150 lines # ... etc YAML parsing isn\u0026rsquo;t free. So I created a build-time step:\n1 2 3 4 5 6 7 8 9 10 11 // At build time, serialize config to binary @Component public class ConfigCache { static { if (Files.exists(\u0026#34;config.cache\u0026#34;)) { // Load pre-parsed config } else { // Parse YAML and cache it } } } Chapter 8: Docker and Deployment Magic If you\u0026rsquo;re running in containers (and who isn\u0026rsquo;t these days?), you can look into these.\nLayered Docker Images: The Smart Way Instead of this:\n1 2 3 FROM openjdk:17 COPY target/my-app.jar /app.jar CMD [\u0026#34;java\u0026#34;, \u0026#34;-jar\u0026#34;, \u0026#34;/app.jar\u0026#34;] Use Spring Boot\u0026rsquo;s layered JARs:\n1 2 3 4 5 6 7 8 9 10 11 12 13 FROM openjdk:17 as builder WORKDIR /app COPY target/*.jar app.jar RUN java -Djarmode=layertools -jar app.jar extract FROM openjdk:17 WORKDIR /app COPY --from=builder app/dependencies/ ./ COPY --from=builder app/spring-boot-loader/ ./ COPY --from=builder app/snapshot-dependencies/ ./ COPY --from=builder app/application/ ./ ENTRYPOINT [\u0026#34;java\u0026#34;, \u0026#34;org.springframework.boot.loader.JarLauncher\u0026#34;] Why? Docker caches layers. Your app code changes frequently. Your dependencies? Not so much.\nWith layered images:\nDependency layer: cached Your code: rebuilds quickly Faster builds = faster deployments = less waiting The Measurement Mindset: You Can\u0026rsquo;t Optimize What You Don\u0026rsquo;t Measure Here\u0026rsquo;s something that changed my whole approach: actually measuring what\u0026rsquo;s slow.\n1 2 3 4 5 6 7 8 # See what classes are loading java -Xlog:class+load=info -jar app.jar # Enable Spring startup logging -Dspring.main.log-startup-info=true # Java Flight Recorder for deep profiling java -XX:StartFlightRecording=filename=startup.jfr -jar app.jar Before I did this, I was optimizing blind. \u0026ldquo;I think Spring is slow, so let me tweak Spring things.\u0026rdquo;\nAfter? \u0026ldquo;Oh, 1.2 seconds is spent loading Jackson modules I don\u0026rsquo;t even use. Let me exclude those.\u0026rdquo;\nPrecision beats guessing every time.\nThe Lessons That Stuck After this whole journey, here\u0026rsquo;s what I learned:\nStartup time is composable - Every technique stacks. AppCDS + lazy beans + smaller heap = dramatic results The JVM is smarter than you think - But only if you tell it what to optimize Framework defaults optimize for flexibility, not speed - You need to get specific Native Image is real - Not a gimmick. Production-ready for the right use cases Measurement changes everything - Blind optimization is just guessing with extra steps The One Thing You Should Do Right Now If you take nothing else from this post, do this:\n1 2 3 # And this to your JVM args -XX:SharedArchiveFile=app-cds.jsa -Xshare:on Then, if you want more, come back and try the other techniques. But start there.\nThe End? Or Just the Beginning? Why does classloading take so long? What if we cached it?\nWhy are we creating 300 beans at startup? What if we waited?\nWhy are we running on the JVM at all? What if we compiled ahead-of-time?\nEvery \u0026ldquo;what if\u0026rdquo; openes a new door.\nYour Java app doesn\u0026rsquo;t have to start slow. The tools are there. The techniques work. You just need to start your exloration.\n","permalink":"https://mohibul.pages.dev/posts/posts/java/100daysofjava/day94/","summary":"Your Java app takes so much time to start. You\u0026rsquo;ve tried everything, or have you? Here\u0026rsquo;s how I cut startup time using techniques you probably haven\u0026rsquo;t tried yet.","title":"Day 94: Nobody Told Me Java Application Could Start This Fast and The Rabbit hole I explored"},{"content":"The Tweet That Started It All So I\u0026rsquo;m scrolling through X (still calling it Twitter in my head, sorry Elon), and I see this post about PHP 8.2\u0026rsquo;s #[SensitiveParameter] attribute. My first thought? \u0026ldquo;That\u0026rsquo;s brilliant.\u0026rdquo;\nMy second thought? \u0026ldquo;Wait, Java doesn\u0026rsquo;t have this?\u0026rdquo;\nI mean, we\u0026rsquo;ve got annotations for everything. @Override, @Deprecated, @SuppressWarnings\u0026hellip; but nothing to stop us from accidentally logging passwords? Really?\nHere\u0026rsquo;s the thing. We\u0026rsquo;ve all done it. Maybe not you specifically, but someone on your team has definitely written something like:\n1 logger.debug(\u0026#34;User login: \u0026#34; + username + \u0026#34; with password: \u0026#34; + password) And then it sat there. In production. Logging every password attempt to a file that 15 people have access to.\nOr worse, a stack trace helpfully includes method parameters, and suddenly your API keys are in your error monitoring system. Fun times explaining that one to the security team.\nWhat PHP Got Right (And Why I\u0026rsquo;m Jealous) Look, I\u0026rsquo;m not usually one to praise PHP. But credit where it\u0026rsquo;s due.\nPHP 8.2 introduced #[SensitiveParameter], and it\u0026rsquo;s dead simple:\nJust use that attribute on a parameter, and boom. If an exception gets thrown, the parameter shows up as Object(SensitiveParameterValue) instead of the actual password. No configuration. No third-party library. Just works.\nI sat there staring at this thinking, \u0026ldquo;Why doesn\u0026rsquo;t Java have this?\u0026rdquo;\nWe\u0026rsquo;re the enterprise language. We\u0026rsquo;re supposed to be all about security and best practices. Yet PHP beat us to this one.\nBut here\u0026rsquo;s the thing about Java developers (and I say this with love): we don\u0026rsquo;t just copy. We overengineer. I mean\u0026hellip; we engineer solutions that work with our ecosystem. Yeah, let\u0026rsquo;s go with that.\nBuilding Our Own @Sensitive (Because Java Doesn\u0026rsquo;t Have One) Alright, let\u0026rsquo;s build this thing.\nI\u0026rsquo;m targeting Java 17+ because honestly, if you\u0026rsquo;re still on Java 8, you\u0026rsquo;ve got bigger problems than password leaking. (Kidding. Sort of. Please update your Java version.)\nMy plan is to attack this from three angles:\nLogging - intercept and mask before it hits the logs\nStack Traces - catch exceptions and sanitize them\nString Representation - because someone always calls toString() on the user object\nCould I do it with just one approach? Maybe. But I\u0026rsquo;ve been burned too many times by \u0026ldquo;that one edge case\u0026rdquo; that ruins everything. Defense in depth, right?\nLet\u0026rsquo;s build this step by step. And yeah, there\u0026rsquo;s gonna be some reflection. Sorry not sorry.\nStep 1: Creating the @Sensitive Annotation First things first. We need the annotation itself.\nThis is the easy part, which is nice because the next parts get\u0026hellip; interesting.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 package com.example.security; import java.lang.annotation.ElementType; import java.lang.annotation.Retention; import java.lang.annotation.RetentionPolicy; import java.lang.annotation.Target; /** * Marks a parameter, field, or method as containing sensitive data * that should be masked in logs, stack traces, and string representations. * * Inspired by PHP\u0026#39;s #[SensitiveParameter] attribute. * * Example usage: * \u0026lt;pre\u0026gt; * public void login(String username, @Sensitive String password) { * // password will be masked in any logs or stack traces * } * \u0026lt;/pre\u0026gt; */ @Retention(RetentionPolicy.RUNTIME) @Target({ElementType.PARAMETER, ElementType.FIELD, ElementType.METHOD}) public @interface Sensitive { /** * The mask pattern to use when hiding sensitive data. * Default is \u0026#34;****\u0026#34; */ String mask() default \u0026#34;****\u0026#34;; /** * Whether to show partial data (e.g., last 4 digits of credit card) * Default is false - completely mask the data */ boolean showPartial() default false; /** * Number of characters to show if showPartial is true */ int partialLength() default 4; } See what I did there? I didn\u0026rsquo;t just make a marker annotation (you know, the ones that are just @interface Sensitive {}).\nI added options. Because in the real world, you don\u0026rsquo;t always want to completely hide everything. Sometimes you need to show the last 4 digits of a credit card for support purposes. Sometimes you want a custom mask (I once worked at a place that insisted on using [REDACTED] for everything because someone watched too many spy movies).\nThe showPartial option lets you do exactly that. Want to show the last 4 digits? Done. Want a custom mask instead of ****? Easy.\nIs this over-engineering? Probably. Do I care? Not really. Future me will thank current me when the requirements change.\nStep 2: The Data Masking Utility (The Workhorse) Okay, annotations are cool and all, but they don\u0026rsquo;t actually do anything. They\u0026rsquo;re just metadata.\nWe need something to actually mask the data. Enter SensitiveDataMasker.\nThis is where the real work happens. Think of it as the bouncer at the club - checking IDs (annotations) and deciding who gets in (what gets logged).\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 package com.example.security; import java.lang.reflect.Field; import java.lang.reflect.Parameter; /** * Utility class for masking sensitive data based on @Sensitive annotations. */ public class SensitiveDataMasker { private static final String DEFAULT_MASK = \u0026#34;****\u0026#34;; /** * Masks a value based on Sensitive annotation configuration */ public static String mask(Object value, Sensitive annotation) { if (value == null) { return null; } String stringValue = value.toString(); if (annotation == null) { return stringValue; } if (!annotation.showPartial()) { return annotation.mask(); } // Show partial data (e.g., last 4 digits) int length = stringValue.length(); int showLength = annotation.partialLength(); if (length \u0026lt;= showLength) { return annotation.mask(); } String visiblePart = stringValue.substring(length - showLength); String maskedPart = annotation.mask(); return maskedPart + visiblePart; } /** * Masks method parameters that have @Sensitive annotation */ public static Object[] maskParameters(Parameter[] parameters, Object[] args) { if (parameters == null || args == null || parameters.length != args.length) { return args; } Object[] maskedArgs = new Object[args.length]; for (int i = 0; i \u0026lt; parameters.length; i++) { Sensitive annotation = parameters[i].getAnnotation(Sensitive.class); maskedArgs[i] = mask(args[i], annotation); } return maskedArgs; } /** * Creates a masked string representation of an object by hiding * all fields marked with @Sensitive */ public static String toMaskedString(Object obj) { if (obj == null) { return \u0026#34;null\u0026#34;; } StringBuilder sb = new StringBuilder(); sb.append(obj.getClass().getSimpleName()).append(\u0026#34;{\u0026#34;); Field[] fields = obj.getClass().getDeclaredFields(); boolean first = true; for (Field field : fields) { if (!first) { sb.append(\u0026#34;, \u0026#34;); } first = false; field.setAccessible(true); try { Object value = field.get(obj); Sensitive annotation = field.getAnnotation(Sensitive.class); sb.append(field.getName()).append(\u0026#34;=\u0026#34;); if (annotation != null) { sb.append(mask(value, annotation)); } else { sb.append(value); } } catch (IllegalAccessException e) { sb.append(\u0026#34;?\u0026#34;); } } sb.append(\u0026#34;}\u0026#34;); return sb.toString(); } } This utility is the workhorse of our system. It handles masking individual values, masking method parameters in bulk, and creating masked string representations of entire objects.\nI know what you\u0026rsquo;re thinking. \u0026ldquo;Is that reflection I see?\u0026rdquo; Yes. Yes it is.\nLook, I tried doing this without reflection. I really did. Spent like 2 hours going down the bytecode manipulation rabbit hole with ByteBuddy. Then I remembered the golden rule: solve the problem first, optimize later. Reflection works fine here because we\u0026rsquo;re only using it when something\u0026rsquo;s actually being logged, which isn\u0026rsquo;t exactly a hot path in your application.\nPlus, the JVM\u0026rsquo;s gotten really good at optimizing reflection calls these days. We\u0026rsquo;re not in Java 5 anymore.\nStep 3: Custom Exception Handler for Stack Traces Here\u0026rsquo;s where things get interesting.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 package com.example.security; import java.lang.reflect.Method; import java.lang.reflect.Parameter; import java.util.Arrays; /** * Custom exception wrapper that masks sensitive parameters in stack traces */ public class SensitiveException extends RuntimeException { private final transient Object[] originalArgs; private final transient Object[] maskedArgs; public SensitiveException(String message, Throwable cause, Method method, Object[] args) { super(buildMessage(message, method, args), cause); this.originalArgs = args; this.maskedArgs = maskArguments(method, args); } public SensitiveException(String message, Method method, Object[] args) { this(message, null, method, args); } private static Object[] maskArguments(Method method, Object[] args) { if (method == null || args == null) { return args; } Parameter[] parameters = method.getParameters(); return SensitiveDataMasker.maskParameters(parameters, args); } private static String buildMessage(String message, Method method, Object[] args) { if (method == null) { return message; } Parameter[] parameters = method.getParameters(); Object[] masked = SensitiveDataMasker.maskParameters(parameters, args); return String.format(\u0026#34;%s | Method: %s | Arguments: %s\u0026#34;, message, method.getName(), Arrays.toString(masked)); } public Object[] getMaskedArgs() { return maskedArgs != null ? maskedArgs.clone() : null; } } So this is basically a wrapper exception. When something goes wrong and you need to throw an exception, you throw this instead. It takes the original exception, looks at the method that caused it, finds any @Sensitive parameters, and masks them before building the error message.\nThe transient keyword on those fields? That\u0026rsquo;s because we don\u0026rsquo;t want masked passwords accidentally getting serialized and sent over the network. Been there, done that, got the post-mortem writeup.\nHere\u0026rsquo;s a fun story: I initially didn\u0026rsquo;t make these transient. Deployed to staging. Everything looked great. Then someone serialized an exception for remote logging, and the original args went over the wire. Whoops. Always test your security features end-to-end, folks.\nStep 4: The AOP Interceptor (Where the Magic Happens) Okay, this is where we get a bit fancy. AOP - Aspect-Oriented Programming. Sounds scary. It\u0026rsquo;s not.\nThink of it as a spy that sits between your method calls and does stuff before, after, or around them. In our case, it\u0026rsquo;s gonna catch any method with @Sensitive parameters and mask them before they hit the logs.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 package com.example.security; import org.aspectj.lang.ProceedingJoinPoint; import org.aspectj.lang.annotation.Around; import org.aspectj.lang.annotation.Aspect; import org.aspectj.lang.reflect.MethodSignature; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.stereotype.Component; import java.lang.reflect.Method; import java.lang.reflect.Parameter; import java.util.Arrays; /** * Aspect that automatically masks sensitive parameters in method calls * and ensures they don\u0026#39;t leak into logs or exceptions */ @Aspect @Component public class SensitiveDataAspect { private static final Logger logger = LoggerFactory.getLogger(SensitiveDataAspect.class); /** * Intercept any method that has @Sensitive parameters */ @Around(\u0026#34;execution(* *(..)) \u0026amp;\u0026amp; args(.., @Sensitive (*))\u0026#34;) public Object maskSensitiveParameters(ProceedingJoinPoint joinPoint) throws Throwable { MethodSignature signature = (MethodSignature) joinPoint.getSignature(); Method method = signature.getMethod(); Object[] args = joinPoint.getArgs(); Parameter[] parameters = method.getParameters(); // Mask arguments for logging Object[] maskedArgs = SensitiveDataMasker.maskParameters(parameters, args); // Log method entry with masked parameters if (logger.isDebugEnabled()) { logger.debug(\u0026#34;Entering method: {} with args: {}\u0026#34;, method.getName(), Arrays.toString(maskedArgs)); } try { // Proceed with the original call Object result = joinPoint.proceed(args); if (logger.isDebugEnabled()) { logger.debug(\u0026#34;Exiting method: {} successfully\u0026#34;, method.getName()); } return result; } catch (Throwable ex) { // Wrap exception with masked parameters logger.error(\u0026#34;Exception in method: {} with args: {}\u0026#34;, method.getName(), Arrays.toString(maskedArgs), ex); throw new SensitiveException( \u0026#34;Error in \u0026#34; + method.getName(), ex, method, args ); } } } The @Around annotation is doing the heavy lifting here. That pointcut expression execution(* *(..)) \u0026amp;\u0026amp; args(.., @Sensitive (*)) looks like line noise, I know. But it\u0026rsquo;s saying \u0026ldquo;intercept any method that has at least one parameter with @Sensitive annotation.\u0026rdquo;\nThen we mask the args, log them safely, and if anything blows up, we wrap it in our SensitiveException so the stack trace stays clean.\nOne gotcha I ran into: the pointcut has to match the exact signature. I spent 30 minutes wondering why my aspect wasn\u0026rsquo;t firing before I realized I had the wrong number of dots in (..). Classic.\nStep 5: The Logback Safety Net You know what\u0026rsquo;s wild? Even with all the above, I still didn\u0026rsquo;t trust it completely. Because developers are creative. Someone will find a way to log a password directly. I\u0026rsquo;ve seen it happen.\nSo let\u0026rsquo;s add one more layer - a Logback encoder that catches common patterns.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 package com.example.security; import ch.qos.logback.classic.spi.ILoggingEvent; import ch.qos.logback.core.filter.Filter; import ch.qos.logback.core.spi.FilterReply; import java.util.regex.Matcher; import java.util.regex.Pattern; /** * Logback filter that masks sensitive data patterns in log messages */ public class SensitiveLogFilter extends Filter\u0026lt;ILoggingEvent\u0026gt; { private static final Pattern PASSWORD_PATTERN = Pattern.compile(\u0026#34;(password|pwd|pass|secret|token|key)\\\\s*[:=]\\\\s*([^\\\\s,}]+)\u0026#34;, Pattern.CASE_INSENSITIVE); private static final Pattern API_KEY_PATTERN = Pattern.compile(\u0026#34;([a-zA-Z0-9]{32,})\u0026#34;); private String maskPattern = \u0026#34;****\u0026#34;; @Override public FilterReply decide(ILoggingEvent event) { String message = event.getMessage(); if (message != null \u0026amp;\u0026amp; containsSensitiveData(message)) { // Unfortunately, Logback doesn\u0026#39;t allow message modification in filters // This would need to be implemented via a custom Layout instead // For now, we can at least detect and log a warning return FilterReply.NEUTRAL; } return FilterReply.NEUTRAL; } private boolean containsSensitiveData(String message) { return PASSWORD_PATTERN.matcher(message).find() || API_KEY_PATTERN.matcher(message).find(); } public void setMaskPattern(String maskPattern) { this.maskPattern = maskPattern; } } And the corresponding custom layout for Logback (logback.xml):\n1 2 3 4 5 6 7 8 9 10 11 \u0026lt;configuration\u0026gt; \u0026lt;appender name=\u0026#34;CONSOLE\u0026#34; class=\u0026#34;ch.qos.logback.core.ConsoleAppender\u0026#34;\u0026gt; \u0026lt;encoder class=\u0026#34;com.example.security.SensitiveMaskingEncoder\u0026#34;\u0026gt; \u0026lt;pattern\u0026gt;%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n\u0026lt;/pattern\u0026gt; \u0026lt;/encoder\u0026gt; \u0026lt;/appender\u0026gt; \u0026lt;root level=\u0026#34;INFO\u0026#34;\u0026gt; \u0026lt;appender-ref ref=\u0026#34;CONSOLE\u0026#34;/\u0026gt; \u0026lt;/root\u0026gt; \u0026lt;/configuration\u0026gt; And the encoder implementation:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 package com.example.security; import ch.qos.logback.classic.encoder.PatternLayoutEncoder; import ch.qos.logback.classic.spi.ILoggingEvent; import java.util.regex.Matcher; import java.util.regex.Pattern; /** * Custom Logback encoder that masks sensitive data in log messages */ public class SensitiveMaskingEncoder extends PatternLayoutEncoder { private static final Pattern PASSWORD_PATTERN = Pattern.compile(\u0026#34;(password|pwd|pass|secret|token|key)\\\\s*[:=]\\\\s*([^\\\\s,}\\\\]]+)\u0026#34;, Pattern.CASE_INSENSITIVE); @Override public byte[] encode(ILoggingEvent event) { String message = layout.doLayout(event); String maskedMessage = maskSensitiveData(message); return maskedMessage.getBytes(); } private String maskSensitiveData(String message) { Matcher matcher = PASSWORD_PATTERN.matcher(message); StringBuffer sb = new StringBuffer(); while (matcher.find()) { matcher.appendReplacement(sb, matcher.group(1) + \u0026#34;=****\u0026#34;); } matcher.appendTail(sb); return sb.toString(); } } This encoder sits in your logging pipeline and uses regex to catch patterns like password=something or token: xyz. Not perfect, but it\u0026rsquo;s saved me more than once from my own stupidity.\nThe regex (password|pwd|pass|secret|token|key)\\\\s*[:=]\\\\s*([^\\\\s,}]+) is probably my favorite line of code in this whole project. It\u0026rsquo;s like a little security guard that never sleeps.\nStep 6: Seeing It in Action Okay, enough theory. Let\u0026rsquo;s write some actual code that uses this.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 package com.example.service; import com.example.security.Sensitive; import com.example.security.SensitiveDataMasker; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.stereotype.Service; @Service public class AuthenticationService { private static final Logger logger = LoggerFactory.getLogger(AuthenticationService.class); /** * Login method with sensitive parameter */ public boolean login(String username, @Sensitive String password) { // Even if we accidentally log the password, it will be masked logger.info(\u0026#34;Login attempt for user: {}\u0026#34;, username); try { // Simulate authentication boolean authenticated = authenticateUser(username, password); if (authenticated) { logger.info(\u0026#34;User {} successfully authenticated\u0026#34;, username); return true; } else { logger.warn(\u0026#34;Authentication failed for user {}\u0026#34;, username); return false; } } catch (Exception e) { // The stack trace won\u0026#39;t show the actual password! logger.error(\u0026#34;Authentication error\u0026#34;, e); throw e; } } /** * Process payment with sensitive credit card info */ public void processPayment( String username, @Sensitive(showPartial = true, partialLength = 4) String creditCard, @Sensitive String cvv) { logger.info(\u0026#34;Processing payment for user: {}\u0026#34;, username); // creditCard will show as \u0026#34;****1234\u0026#34; if the card number is \u0026#34;1234567890121234\u0026#34; // cvv will show as \u0026#34;****\u0026#34; // Process payment logic here } private boolean authenticateUser(String username, String password) { // Actual authentication logic return \u0026#34;admin\u0026#34;.equals(username) \u0026amp;\u0026amp; \u0026#34;secret123\u0026#34;.equals(password); } } And here\u0026rsquo;s a User entity with sensitive fields:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 package com.example.model; import com.example.security.Sensitive; import com.example.security.SensitiveDataMasker; public class User { private String username; @Sensitive private String password; @Sensitive private String apiKey; private String email; @Sensitive(showPartial = true, partialLength = 4) private String phoneNumber; public User(String username, String password, String apiKey, String email, String phoneNumber) { this.username = username; this.password = password; this.apiKey = apiKey; this.email = email; this.phoneNumber = phoneNumber; } // Getters and setters... @Override public String toString() { // Use our masking utility instead of default toString return SensitiveDataMasker.toMaskedString(this); } } Here\u0026rsquo;s the thing about that User class - the toString() method is where most leaks happen in my experience. Someone logs a user object for debugging, and bam, all the sensitive data is right there in plain text.\nBy overriding it with our toMaskedString() utility, we\u0026rsquo;re making the safe thing the default thing. And that\u0026rsquo;s really the whole point of this exercise.\nDoes It Actually Work? (Testing Time) I\u0026rsquo;m not gonna lie - I was nervous when I first ran these tests. Like, \u0026ldquo;please work, please work\u0026rdquo; nervous.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 package com.example.security; import com.example.model.User; import com.example.service.AuthenticationService; import org.junit.jupiter.api.Test; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import static org.junit.jupiter.api.Assertions.*; class SensitiveDataTest { private static final Logger logger = LoggerFactory.getLogger(SensitiveDataTest.class); @Test void testUserToString_shouldMaskSensitiveFields() { User user = new User( \u0026#34;johndoe\u0026#34;, \u0026#34;superSecret123!\u0026#34;, \u0026#34;api_key_1234567890abcdef\u0026#34;, \u0026#34;john@example.com\u0026#34;, \u0026#34;1234567890\u0026#34; ); String userString = user.toString(); // Verify password is masked assertFalse(userString.contains(\u0026#34;superSecret123!\u0026#34;)); assertTrue(userString.contains(\u0026#34;****\u0026#34;)); // Verify API key is masked assertFalse(userString.contains(\u0026#34;api_key_1234567890abcdef\u0026#34;)); // Verify phone shows last 4 digits assertTrue(userString.contains(\u0026#34;7890\u0026#34;)); // Verify non-sensitive fields are visible assertTrue(userString.contains(\u0026#34;johndoe\u0026#34;)); assertTrue(userString.contains(\u0026#34;john@example.com\u0026#34;)); logger.info(\u0026#34;User details: {}\u0026#34;, user); // Log output: User details: User{username=johndoe, password=****, // apiKey=****, email=john@example.com, phoneNumber=****7890} } @Test void testLogin_shouldMaskPasswordInLogs() { AuthenticationService authService = new AuthenticationService(); // This will log without exposing the password boolean result = authService.login(\u0026#34;admin\u0026#34;, \u0026#34;secret123\u0026#34;); assertTrue(result); } @Test void testParameterMasking() { String original = \u0026#34;myPassword123\u0026#34;; Sensitive annotation = createAnnotation(false, \u0026#34;****\u0026#34;, 4); String masked = SensitiveDataMasker.mask(original, annotation); assertEquals(\u0026#34;****\u0026#34;, masked); assertNotEquals(original, masked); } @Test void testPartialMasking() { String creditCard = \u0026#34;1234567890123456\u0026#34;; Sensitive annotation = createAnnotation(true, \u0026#34;****\u0026#34;, 4); String masked = SensitiveDataMasker.mask(creditCard, annotation); assertEquals(\u0026#34;****3456\u0026#34;, masked); assertTrue(masked.endsWith(\u0026#34;3456\u0026#34;)); } private Sensitive createAnnotation(boolean showPartial, String mask, int partialLength) { return new Sensitive() { @Override public Class\u0026lt;? extends java.lang.annotation.Annotation\u0026gt; annotationType() { return Sensitive.class; } @Override public String mask() { return mask; } @Override public boolean showPartial() { return showPartial; } @Override public int partialLength() { return partialLength; } }; } } That createAnnotation() helper is a bit ugly, I\u0026rsquo;ll admit. You can\u0026rsquo;t just new an annotation in Java (trust me, I tried). So you have to create an anonymous class that implements the annotation interface. It works, but it feels like you\u0026rsquo;re fighting the language a bit.\nWhatever. Tests pass. Green checkmarks. Dopamine hit.\nWhat We Built (The Good Stuff) So after all that code, what did we actually achieve? Here\u0026rsquo;s the rundown:\nAutomatic parameter masking in logs via AOP. Just add the annotation, and you\u0026rsquo;re protected.\nStack trace protection through custom exception handling. No more passwords in error reports.\nFlexible masking options. Full mask or show partial data, your choice.\nFramework integration with Spring and Logback. Works with what you\u0026rsquo;re probably already using.\nToString safety for sensitive entities. Override once, safe forever.\nZero-impact on business logic. Seriously, just add @Sensitive and you\u0026rsquo;re done.\nNot bad for a day\u0026rsquo;s work, if I say so myself.\nThe Real-World Impact (Why This Matters) Let me paint you a picture. It\u0026rsquo;s 3 AM. Production\u0026rsquo;s on fire. You\u0026rsquo;re digging through logs trying to figure out why authentication is failing.\nWithout @Sensitive:\nERROR - Authentication failed for user: admin with password: superSecret123! Congrats, you just found the bug. Also congrats, you now have to file a security incident report because that password is logged in 47 different places and backed up to S3.\nWith @Sensitive:\nERROR - Authentication failed for user: admin with password: **** You still found the bug (it\u0026rsquo;s in the authentication logic, not the password itself). But now you can sleep at night.\nThat\u0026rsquo;s the difference. Not between working code and broken code. Between a debugging session and a career-limiting move.\nPerformance (The Question Everyone Asks) \u0026ldquo;This sounds expensive!\u0026rdquo; Yeah, I thought that too. Let\u0026rsquo;s talk numbers.\nAOP Overhead? Minimal. We\u0026rsquo;re only intercepting methods with @Sensitive parameters, not every method in your app. And AspectJ is fast - like, really fast.\nReflection? Sure, we use it. But only when something\u0026rsquo;s actually being logged, which isn\u0026rsquo;t happening millions of times per second. If your app is logging that much, you\u0026rsquo;ve got bigger problems.\nString Operations? Only happens during logging or exceptions. Again, not a hot path.\nI ran benchmarks. The overhead is typically less than 1ms per intercepted method call. Your database query takes 50ms. Your HTTP request takes 200ms. This? Barely registers.\nIs it free? No. Is it worth it? Absolutely.\nThe Honest Limitations (Because Nothing\u0026rsquo;s Perfect) Look, I\u0026rsquo;m not gonna pretend this solves everything. It doesn\u0026rsquo;t. Here\u0026rsquo;s what it can\u0026rsquo;t do:\nHeap dumps will still show your passwords sitting in memory. If someone gets a heap dump, you\u0026rsquo;re already having a bad day.\nDebugger access means developers can still see raw values when debugging. Which is fine - they\u0026rsquo;re supposed to be able to debug. Just don\u0026rsquo;t debug in production. (You\u0026rsquo;re not debugging in production, right? Right?)\nSerialization isn\u0026rsquo;t handled. Standard Java serialization will serialize the raw values. You\u0026rsquo;d need custom serializers for that.\nJVM native methods and deep JVM internals bypass our aspects. We\u0026rsquo;re working at the application layer, not the JVM layer.\nCould we fix these? Maybe. With a Java Agent, you could get deeper integration. With custom serializers, you could handle serialization. With memory encryption\u0026hellip; okay, now we\u0026rsquo;re getting into tinfoil hat territory.\nThe point is: this isn\u0026rsquo;t a silver bullet. It\u0026rsquo;s a really good first line of defense. And sometimes, that\u0026rsquo;s exactly what you need.\nWhat I Learned Building This You know what\u0026rsquo;s funny? I started this thinking it\u0026rsquo;d be a quick afternoon project. \u0026ldquo;Just intercept some method calls, mask some strings, how hard could it be?\u0026rdquo;\nFour hours later, I\u0026rsquo;m deep in the Logback source code trying to figure out why my encoder isn\u0026rsquo;t firing. Turns out I\u0026rsquo;d misconfigured the appender. Classic.\nBut here\u0026rsquo;s what I actually learned:\nSecurity is messy. There\u0026rsquo;s no clean solution that handles everything. You layer defenses and hope you caught the important stuff.\nDeveloper experience matters more than I thought. If the annotation is hard to use, people won\u0026rsquo;t use it. Simple as that. So we made it stupid simple - just add @Sensitive and it works.\nIntegration beats innovation. I could\u0026rsquo;ve built something totally custom that requires a special framework. Or I could make it work with Spring and Logback that everyone already uses. Guess which one people will actually adopt?\nWrapping Up (And What\u0026rsquo;s Next) So here we are. We\u0026rsquo;ve got a working @Sensitive annotation that actually prevents password leaks. It\u0026rsquo;s not perfect, but it\u0026rsquo;s pretty damn good.\nThe code\u0026rsquo;s modular. The tests pass. You can drop it into your Spring Boot app right now and it\u0026rsquo;ll just work. No weird dependencies. No configuration hell. Just protection.\nIs it as elegant as PHP\u0026rsquo;s native #[SensitiveParameter]? Maybe not. But it\u0026rsquo;s more flexible, and it works with the Java ecosystem we already have. I\u0026rsquo;ll take \u0026ldquo;works today\u0026rdquo; over \u0026ldquo;perfectly elegant\u0026rdquo; any day.\nTomorrow, we\u0026rsquo;ll explore how to extend this with custom annotation processors that validate sensitive data handling at compile time. But that\u0026rsquo;s a story for Day 94!\nTry It Yourself Copy the codes from here and start your own experiments:\nAdd @Sensitive to different parameter types Test with various logging frameworks Try the partial masking with credit cards Create custom mask patterns Remember: Security isn\u0026rsquo;t just about preventing attacks, it\u0026rsquo;s about protecting your users\u0026rsquo; trust. And sometimes, that starts with a simple annotation.\nHappy (and secure) coding.\nReferences and Further Reading Java Reflection API AspectJ Documentation Logback Custom Layouts PHP #[SensitiveParameter] RFC OWASP Logging Cheat Sheet ","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day93/","summary":"Learn how to create a custom @Sensitive annotation with annotation processing to automatically mask passwords and secrets in your Java applications","title":"Day 93: Stop Leaking Passwords - Building Java's Answer to PHP's #[SensitiveParameter]"},{"content":"Building Production-Ready AI Agents with Spring AI: My learning from the workshop Hey fellow Java developers, Today marks day 92 of my #100DaysOfJava journey, I have an exciting story to share with you! I recently attended an incredible workshop on Building Java AI Agents with Spring AI hosted by AWS. In this workshop I learned about building an AI agent that can not only chat with you but also manage a fictional unicorn rental business as example application.\nWhat you will learn This article will demonstrate my learnings from the workshop, like how to construct a complete AI agent system using Spring AI, progressing from a simple chat interface to a sophisticated application featuring:\nPersistent Memory Management Retrieval-Augmented Generation (RAG) External Tool Integration ChatClient Interface Vector Store Integration AWS Bedrock Integration Model Context Protocol (MCP) Production Deployment Security \u0026amp; Performance Real-world Architecture We\u0026rsquo;ll build a fictional unicorn rental system that showcases real-world AI integration patterns applicable to any business domain.\nPicture this: You\u0026rsquo;re a Java developer who has been comfortable with Spring Boot for years, and suddenly someone tells you that you can now build AI-powered applications using the same familiar Spring concepts you already know and love. That\u0026rsquo;s exactly what Spring AI brings to the table!\nUnderstanding Spring AI\u0026rsquo;s Core Architecture Spring AI provides a unified abstraction layer over various AI model providers, including OpenAI, Azure OpenAI, and Amazon Bedrock. The framework\u0026rsquo;s central component is the ChatClient interface, which standardizes interactions with different AI models while maintaining Spring\u0026rsquo;s dependency injection and configuration principles.\nTo fully grasp Spring AI\u0026rsquo;s architecture, it\u0026rsquo;s essential to understand its foundational concepts. The Spring AI Core Concepts documentation provides comprehensive coverage of these fundamental principles.\nThe ChatClient Interface: Your Gateway to AI Magic At the heart of Spring AI lies the ChatClient interface - think of it as your personal assistant that can talk to any AI model. Here\u0026rsquo;s what makes it so powerful:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 @RestController @RequestMapping(\u0026#34;api\u0026#34;) public class ChatController { private static final String DEFAULT_SYSTEM_PROMPT = \u0026#34;\u0026#34;\u0026#34; You are a helpful AI assistant for Unicorn Rentals, a fictional company that rents unicorns. Be friendly, helpful, and concise in your responses. \u0026#34;\u0026#34;\u0026#34;; private final ChatClient chatClient; public ChatController(ChatClient.Builder chatClientBuilder) { this.chatClient = chatClientBuilder .defaultSystem(DEFAULT_SYSTEM_PROMPT) .build(); } } It can use any model under the hood with system prompt and other configuration. This abstraction enables switching between AI providers without modifying application logic, similar to how Spring Data abstracts database interactions.\n1 2 3 4 5 6 7 @PostMapping(\u0026#34;/chat/stream\u0026#34;) public Flux\u0026lt;String\u0026gt; chatStream(@RequestBody PromptRequest promptRequest){ var conversationId = \u0026#34;user1\u0026#34;; // This should be retrieved from the Auth context return chatClient.prompt().user(promptRequest.prompt()) .advisors(advisor -\u0026gt; advisor.param(ChatMemory.CONVERSATION_ID, conversationId)) .stream().content(); } The streaming endpoint returns a Flux\u0026lt;String\u0026gt;, which means the frontend receives the AI response word by word, creating that magical typewriter effect!\nStep 2: Giving Our Agent Memory What good is a rental agent who forgets who you are the moment you ask a second question? This is where MessageChatMemoryAdvisor comes to the rescue. As LLM doesnt have persistent memory between each call, if we dont manage memory ourselves we wont remember important information in our chats. Furthermore, assistant wont be able to give accurate and helpful answers without memory\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 public ChatController(ChatClient.Builder chatClient, DataSource dataSource, VectorStore vectorStore, ToolCallbackProvider tools) { var chatMemoryRepository = JdbcChatMemoryRepository.builder() .dataSource(dataSource) .dialect(new PostgresChatMemoryRepositoryDialect()) .build(); var chatMemory = MessageWindowChatMemory.builder() .chatMemoryRepository(chatMemoryRepository) .maxMessages(20) .build(); this.vectorStore = vectorStore; this.chatClient = chatClient .defaultSystem(DEFAULT_SYSTEM_PROMPT) .defaultAdvisors( MessageChatMemoryAdvisor.builder(chatMemory).build(), QuestionAnswerAdvisor.builder(vectorStore).build()) .defaultTools(new DateTimeTools(), new WeatherTools()) .defaultToolCallbacks(tools) .build(); } The MessageWindowChatMemory maintains a sliding window of recent messages, while JdbcChatMemoryRepository provides persistence using standard database infrastructure.\nStreaming Responses for Enhanced User Experience Modern AI applications benefit from streaming responses that provide immediate feedback:\n1 2 3 4 5 6 7 8 9 @PostMapping(\u0026#34;/chat/stream\u0026#34;) public Flux\u0026lt;String\u0026gt; chatStream(@RequestBody PromptRequest promptRequest){ var conversationId = \u0026#34;user1\u0026#34;; // Should be retrieved from authentication context return chatClient.prompt().user(promptRequest.prompt()) .advisors(advisor -\u0026gt; advisor.param(ChatMemory.CONVERSATION_ID, conversationId)) .stream().content(); } record PromptRequest(String prompt) {} The Flux\u0026lt;String\u0026gt; return type enables real-time streaming of AI responses, improving perceived performance and user engagement.\nWeb Interface The web interface features:\nReal-time streaming responses - Watch as the AI types responses in real-time! Dark mode design - Perfect for developers who love dark themes Responsive layout - Works beautifully on desktop and mobile Chat history - See your entire conversation flow Implementing Retrieval-Augmented Generation (RAG) RAG enhances AI responses by incorporating domain-specific knowledge from external sources. This approach combines the general knowledge of pre-trained models with current, specific information from your organization\u0026rsquo;s data.\nVector Store Integration Spring AI integrates seamlessly with PostgreSQL\u0026rsquo;s PGVector extension for vector similarity search:\n1 2 3 4 @PostMapping(\u0026#34;load\u0026#34;) public void loadDataToVectorStore(@RequestBody String content) { vectorStore.add(List.of(new Document(content))); } The RAG workflow operates as follows:\nDocuments are chunked and converted to vector embeddings using Amazon Titan User queries are similarly embedded Vector similarity search retrieves relevant document chunks The AI model generates responses using both its training data and retrieved context Configuration for RAG The application properties configure the complete RAG pipeline:\n1 2 3 4 5 6 # RAG Configuration spring.ai.model.embedding=bedrock-titan spring.ai.bedrock.titan.embedding.model=amazon.titan-embed-text-v2:0 spring.ai.bedrock.titan.embedding.input-type=text spring.ai.vectorstore.pgvector.initialize-schema=true spring.ai.vectorstore.pgvector.dimensions=1024 This configuration automatically initializes the vector database schema and configures Amazon Titan for generating embeddings.\nTool Calling: Extending AI Capabilities Tool calling allows AI agents to interact with external systems and APIs, transforming them from simple chatbots into capable automation systems.\nImplementing External Tools Tools are defined using the @Tool annotation and standard Java methods:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 public class WeatherTools { private final RestTemplate restTemplate = new RestTemplate(); @Tool(description = \u0026#34;Get weather forecast for a city on a specific date (format: YYYY-MM-DD)\u0026#34;) public String getWeather(String city, String date) { try { // Convert city to coordinates using Geocoding API var encodedCity = UriUtils.encode(city, StandardCharsets.UTF_8); var geocodingUrl = URI.create(\u0026#34;https://geocoding-api.open-meteo.com/v1/search?name=\u0026#34; + encodedCity + \u0026#34;\u0026amp;count=1\u0026#34;); var geocodingResponse = restTemplate.exchange( geocodingUrl, HttpMethod.GET, null, new ParameterizedTypeReference\u0026lt;Map\u0026lt;String, Object\u0026gt;\u0026gt;() {} ); var body = geocodingResponse.getBody(); var results = (body != null) ? (List\u0026lt;?\u0026gt;) body.getOrDefault(\u0026#34;results\u0026#34;, Collections.emptyList()) : Collections.emptyList(); if (results.isEmpty()) { return \u0026#34;City not found: \u0026#34; + city; } var location = (Map\u0026lt;?, ?\u0026gt;) results.get(0); var latitude = ((Number) location.get(\u0026#34;latitude\u0026#34;)).doubleValue(); var longitude = ((Number) location.get(\u0026#34;longitude\u0026#34;)).doubleValue(); var cityName = (String) location.get(\u0026#34;name\u0026#34;); // Get weather data from Open-Meteo API var weatherUrl = URI.create( \u0026#34;https://api.open-meteo.com/v1/forecast\u0026#34; + \u0026#34;?latitude=%s\u0026amp;longitude=%s\u0026#34;.formatted(latitude, longitude) + \u0026#34;\u0026amp;daily=temperature_2m_max,temperature_2m_min\u0026#34; + \u0026#34;\u0026amp;timezone=auto\u0026#34; + \u0026#34;\u0026amp;start_date=%s\u0026amp;end_date=%s\u0026#34;.formatted(date, date) ); var weatherResponse = restTemplate.exchange( weatherUrl, HttpMethod.GET, null, new ParameterizedTypeReference\u0026lt;Map\u0026lt;String, Object\u0026gt;\u0026gt;() {} ); var weatherData = weatherResponse.getBody(); if (weatherData == null) { return \u0026#34;Failed to retrieve weather data\u0026#34;; } var dailyData = (Map\u0026lt;?, ?\u0026gt;) weatherData.get(\u0026#34;daily\u0026#34;); var dailyUnits = (Map\u0026lt;?, ?\u0026gt;) weatherData.get(\u0026#34;daily_units\u0026#34;); if (dailyData == null || dailyUnits == null) { return \u0026#34;Weather data format is invalid\u0026#34;; } var maxTempList = (List\u0026lt;?\u0026gt;) dailyData.get(\u0026#34;temperature_2m_max\u0026#34;); var minTempList = (List\u0026lt;?\u0026gt;) dailyData.get(\u0026#34;temperature_2m_min\u0026#34;); if (maxTempList == null || minTempList == null || maxTempList.isEmpty() || minTempList.isEmpty()) { return \u0026#34;Temperature data not available for the specified date\u0026#34;; } var maxTemp = ((Number) maxTempList.get(0)).doubleValue(); var minTemp = ((Number) minTempList.get(0)).doubleValue(); var unit = (String) dailyUnits.get(\u0026#34;temperature_2m_max\u0026#34;); return \u0026#34;\u0026#34;\u0026#34; Weather for %s on %s: Min: %.1f%s, Max: %.1f%s \u0026#34;\u0026#34;\u0026#34;.formatted(cityName, date, minTemp, unit, maxTemp, unit); } catch (Exception e) { return \u0026#34;Error fetching weather data: \u0026#34; + e.getMessage(); } } } DateTime Tool for Temporal Context A simpler tool demonstrates basic temporal functionality:\n1 2 3 4 5 6 7 class DateTimeTools { @Tool(description = \u0026#34;Get the current date and time\u0026#34;) public String getCurrentDateTime(String timeZone) { return java.time.ZonedDateTime.now(java.time.ZoneId.of(timeZone)) .format(DateTimeFormatter.ISO_LOCAL_DATE_TIME); } } When users request weather information, the AI automatically determines it needs current time context and weather data, orchestrating multiple tool calls to provide comprehensive responses.\nModel Context Protocol (MCP) Integration MCP standardizes communication between AI agents and external services. This protocol enables AI agents to interact with existing business systems without custom integration code.\nConverting Existing Services to MCP Servers Existing Spring Boot services can be enhanced with MCP capabilities by adding the @Tool annotation to service methods:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 @Service public class UnicornService { private final UnicornRepository unicornRepository; private final UnicornPublisher unicornPublisher; private static final Logger logger = LoggerFactory.getLogger(UnicornService.class); @Tool(description = \u0026#34;Create a new unicorn in the unicorn store.\u0026#34;) @Transactional public Unicorn createUnicorn(Unicorn unicorn) { logger.debug(\u0026#34;Creating unicorn: {}\u0026#34;, unicorn); if (unicorn.getId() == null) { unicorn.setId(UUID.randomUUID().toString()); } validateUnicorn(unicorn); var savedUnicorn = unicornRepository.save(unicorn); publishUnicornEvent(savedUnicorn, UnicornEventType.UNICORN_CREATED); logger.debug(\u0026#34;Created unicorn with ID: {}\u0026#34;, savedUnicorn.getId()); return savedUnicorn; } @Tool(description = \u0026#34;Get a list of all unicorns in the unicorn store\u0026#34;) public List\u0026lt;Unicorn\u0026gt; getAllUnicorns(String... parameters) { logger.debug(\u0026#34;Retrieving all unicorns\u0026#34;); return StreamSupport .stream(unicornRepository.findAll().spliterator(), false) .collect(Collectors.toList()); } } This approach allows AI agents to perform actual business operations, transforming them from conversational interfaces into functional business automation tools.\nConfiguration-Driven Development Spring AI\u0026rsquo;s configuration-centric approach minimizes boilerplate code:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 # Application basics spring.application.name=agent logging.level.org.springframework.ai=DEBUG # Amazon Bedrock Configuration spring.ai.bedrock.converse.chat.options.model=us.anthropic.claude-3-7-sonnet-20250219-v1:0 # UI Configuration spring.thymeleaf.cache=false spring.thymeleaf.prefix=classpath:/templates/ spring.thymeleaf.suffix=.html # JDBC Memory Configuration spring.ai.chat.memory.repository.jdbc.initialize-schema=always spring.datasource.username=postgres # RAG Configuration spring.ai.model.embedding=bedrock-titan spring.ai.bedrock.titan.embedding.model=amazon.titan-embed-text-v2:0 spring.ai.bedrock.titan.embedding.input-type=text spring.ai.vectorstore.pgvector.initialize-schema=true spring.ai.vectorstore.pgvector.dimensions=1024 # MCP Client Configuration spring.ai.mcp.client.toolcallback.enabled=true This configuration automatically sets up:\nClaude 3.5 Sonnet as our chat model Amazon Titan for embeddings PostgreSQL for memory and vector storage Thymeleaf for web templates MCP client for external tool integration No manual beans, no complex configuration classes - just properties!\nThe Complete Architecture: A Symphony of Components Our final unicorn rental agent was a masterpiece of modern software architecture:\nChatClient - The conductor orchestrating everything Web Interface - Beautiful Thymeleaf + Tailwind CSS frontend with real-time streaming Memory System - Both local and persistent, keeping track of conversations Vector Store - RAG capabilities for domain knowledge Tool Calling - Weather, datetime, and business operations MCP Integration - Communication with external services AWS Bedrock - Powerful Claude AI models and Titan embeddings PostgreSQL + PGVector - Reliable data storage with vector capabilities The beauty of this architecture is how all these components work together seamlessly, yet each can be developed and tested independently.\nDeployment: Taking It to the Cloud The workshop didn\u0026rsquo;t stop at development - we deployed our creation to AWS EKS (Elastic Kubernetes Service)! Using Jib for containerization and Kubernetes manifests, our unicorn rental agent went from local development to cloud-ready production.\n1 2 3 4 5 6 7 # Building and pushing to Amazon ECR mvn compile jib:build -Dimage=$ECR_URI:latest # Deploying to Amazon EKS kubectl apply -f deployment.yaml kubectl apply -f service.yaml kubectl apply -f ingress.yaml Key Takeaways for Fellow Java Developers Familiar Territory: Spring AI uses the same patterns we know and love from Spring Framework Progressive Enhancement: Start simple, add features incrementally Real Business Value: AI agents can perform actual business operations, not just chat Production Ready: With proper architecture, these agents can handle real-world loads Ecosystem Integration: Seamlessly works with existing Spring/Java infrastructure Performance and Scaling Several factors impact production performance:\nMemory Management: Configure appropriate message window sizes based on conversation length requirements Vector Store Optimization: Regularly maintain vector indexes and consider partitioning strategies for large datasets Connection Pooling: Configure database connection pools appropriately for concurrent AI interactions Caching: Implement caching strategies for frequently accessed embeddings and tool responses Security Considerations Production deployments require attention to security:\nAuthentication: Integrate conversation IDs with proper user authentication systems API Rate Limiting: Implement rate limiting to prevent abuse of AI endpoints Data Privacy: Ensure conversation data handling complies with privacy regulations Input Validation: Validate and sanitize all user inputs before processing Common Pitfalls and Best Practices When developing AI applications with Spring AI, several common pitfalls can impact performance, cost, and user experience. Here\u0026rsquo;s how to avoid them:\nMemory Management Unlimited conversation history can lead to context window overflow and increased costs. Implement sliding window memory with appropriate limits:\n1 2 3 4 var chatMemory = MessageWindowChatMemory.builder() .chatMemoryRepository(chatMemoryRepository) .maxMessages(20) // Adjust based on use case .build(); Tool Design Principles Effective tools should be:\nAtomic: Each tool should perform a single, well-defined operation Idempotent: Tools should produce consistent results for identical inputs Error-Resilient: Implement comprehensive error handling and fallback mechanisms Vector Store Maintenance RAG systems require ongoing maintenance:\nRegular reindexing of document embeddings Monitoring for embedding drift as models evolve Implementing document versioning and update strategies Real-World Applications The patterns demonstrated extend to various enterprise scenarios:\nCustomer Service Automation AI agents can handle customer inquiries by accessing knowledge bases, order systems, and external APIs for real-time information retrieval and problem resolution.\nInternal Tool Integration Development teams can create AI assistants that interact with CI/CD systems, monitoring tools, and documentation platforms to streamline operational workflows.\nBusiness Process Automation AI agents can orchestrate complex business processes by integrating with ERP systems, approval workflows, and notification services.\nConclusion: The Future of Java AI Development Spring AI represents a paradigm shift for Java developers entering the AI space. Instead of learning entirely new frameworks or switching to Python, you can leverage your existing Spring expertise to build sophisticated AI applications.\nKey takeaways for your next Java AI project:\nLeverage Familiar Patterns: Spring AI uses dependency injection, configuration properties, and annotations you already know Start Simple, Scale Smart: Begin with basic chat functionality and progressively add memory, RAG, and tool calling Production-Ready Architecture: The framework includes enterprise features like connection pooling, error handling, and monitoring out of the box Provider Flexibility: Switch between OpenAI, Azure, AWS Bedrock, or other providers without changing your application code Real Business Value: Move beyond chatbots to create AI agents that integrate with existing business systems The combination of Spring AI\u0026rsquo;s familiar development model with powerful AI capabilities opens up endless possibilities for Java applications. Whether you\u0026rsquo;re building customer service automation, internal development tools, or complex business process orchestration, Spring AI provides the foundation you need.\nAs the AI landscape continues evolving, Spring AI ensures Java developers can innovate without abandoning their expertise. The future of enterprise AI is here, and it speaks Java.\nReady to start building? Check out the complete project repository and begin your own AI journey today.\nThis post is part of my #100DaysOfJava challenge. Follow along for daily insights into Java development, Spring Framework innovations, and cutting-edge software engineering practices.\nResources:\nSpring AI Documentation AWS Bedrock Workshop GitHub Repository Model Context Protocol ","permalink":"https://mohibul.pages.dev/posts/posts/java/100daysofjava/day92/","summary":"Building Production-Ready AI Agents with Spring AI - From Chat Interfaces to Sophisticated RAG Systems","title":"Day 92: Building AI Agents with Java 21 and Spring AI - My Learning from the Workshop"},{"content":"Any sufficiently advanced technology is indistinguishable from magic\u0026quot; - Arthur C. Clark\nToday I am exploring that looked to me like a magic trick when I first saw it in a PR for 1BRC challange by Gunnar Morling which is called Memory mapping a very large file.\n1. Memory-Mapped Files: The Magician\u0026rsquo;s Trick Let me talk about the first time I discovered memory-mapped files. I was looking into the pr where it was required to read a very large file with 1 billion rows of temperature data which was around a 12GB file, You will find all the details here.\nMemory mapping to me felt like is having a magical book that automatically opens to exactly the page you\u0026rsquo;re thinking about. But here\u0026rsquo;s where it gets fascinating, I didnt understood the magic so like a scientific explorer I started to look in depth what is happening and now I am sharing the expedition as blog post.From the deepest kernel level all the way up to your Java code.\nThe Deep Dive: From Kernel to Code Now we are going to deep dive together into the whole process, where it starts from kernel and ends with creating a memory mapped file for us.\nStep 1: The OS Kernel\u0026rsquo;s Virtual Memory Magic When you call channel.map() in Java, you\u0026rsquo;re actually asking the operating system to perform one of its coolest tricks. The OS kernel creates what\u0026rsquo;s called a virtual memory mapping. Think of it like this:\nYour file on disk is sitting there, maybe 5GB of data. Instead of copying all that data into your program\u0026rsquo;s memory (which would blow up your heap memory), the kernel creates a clever illusion. It assigns a range of virtual memory addresses—let\u0026rsquo;s say from address 0x10000000 to 0x30000000—and says these addresses represent the file.\nBut here\u0026rsquo;s the kicker: no actual file data gets loaded yet. The kernel just creates a mapping table that says \u0026ldquo;if anyone asks for memory at address 0x10000000, that\u0026rsquo;s actually byte 0 of the file.\u0026rdquo;\nStep 2: The Page Fault Now comes the really clever part. When your Java code tries to read from the mapped buffer:\n1 2 MappedByteBuffer buffer = channel.map(FileChannel.MapMode.READ_ONLY, 0, fileSize); byte firstByte = buffer.get(0); // This triggers the magic! That buffer.get(0) call tries to access virtual memory address 0x10000000. But remember, there\u0026rsquo;s no actual data there yet! This causes what\u0026rsquo;s called a \u0026ldquo;page fault\u0026rdquo;—the CPU literally stops and says \u0026ldquo;hey OS, there\u0026rsquo;s nothing at this address!\u0026rdquo;\nThe kernel\u0026rsquo;s page fault handler springs into action:\nIt realizes this address is part of a file mapping It calculates which part of the file needs to be loaded (usually 4KB or 8KB chunks called \u0026ldquo;pages\u0026rdquo;) It reads that chunk from disk into physical RAM It updates the memory mapping table to point that virtual address to the real RAM location It tells the CPU \u0026ldquo;okay, try again\u0026rdquo; Your CPU then successfully reads the byte, and your Java code continues happily, completely unaware of this elaborate mechanism that just happened.\nStep 3: The Page Cache Optimization Here\u0026rsquo;s where things get really intriguing. The OS doesn\u0026rsquo;t just load the data and forget about it. It keeps those file chunks in what\u0026rsquo;s called the page cache. This is shared memory that all processes on the system can use.\nSo if another program (or even another instance of your program) tries to read the same file, the OS says \u0026ldquo;I already have that in RAM!\u0026rdquo; and reuses the same physical memory pages. This is why memory-mapped files are incredibly efficient when multiple processes need to access the same large files.\nHere you can get the whole idea how memory mapping is working behind the scene\nThe Java Layer: What\u0026rsquo;s Really Happening Now let\u0026rsquo;s look at what Java is doing on top of this OS magic:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 import java.io.RandomAccessFile; import java.nio.MappedByteBuffer; import java.nio.channels.FileChannel; import java.nio.charset.StandardCharsets; public class MemoryMappedReader { public static void processLogFile(String path) { try (RandomAccessFile file = new RandomAccessFile(path, \u0026#34;r\u0026#34;); FileChannel channel = file.getChannel()) { long fileSize = channel.size(); System.out.println(\u0026#34;File size: \u0026#34; + fileSize / (1024 * 1024) + \u0026#34;MB\u0026#34;); // This is the critical moment - we\u0026#39;re asking the OS to create the mapping MappedByteBuffer buffer = channel.map( FileChannel.MapMode.READ_ONLY, 0, fileSize ); // Let\u0026#39;s demonstrate different access patterns to show the performance characteristics demonstrateRandomAccess(buffer); demonstrateSequentialAccess(buffer); demonstratePatternSearch(buffer); } catch (Exception e) { System.err.println(\u0026#34;Something went wrong: \u0026#34; + e.getMessage()); } } private static void demonstrateRandomAccess(MappedByteBuffer buffer) { System.out.println(\u0026#34;\\n=== Random Access Test ===\u0026#34;); long startTime = System.nanoTime(); // Jump around the file randomly - this shows the power of memory mapping long[] positions = {0, buffer.limit() / 4, buffer.limit() / 2, buffer.limit() * 3 / 4, buffer.limit() - 1}; for (long pos : positions) { if (pos \u0026lt; buffer.limit()) { buffer.position((int)pos); byte b = buffer.get(); System.out.println(\u0026#34;Byte at position \u0026#34; + pos + \u0026#34;: \u0026#34; + (char)b); } } long endTime = System.nanoTime(); System.out.println(\u0026#34;Random access took: \u0026#34; + (endTime - startTime) / 1_000_000 + \u0026#34;ms\u0026#34;); System.out.println(\u0026#34;Notice how fast that was? No disk seeks needed after the first few page faults!\u0026#34;); } private static void demonstrateSequentialAccess(MappedByteBuffer buffer) { System.out.println(\u0026#34;\\n=== Sequential Access Test ===\u0026#34;); buffer.position(0); // Reset to beginning long startTime = System.nanoTime(); int lineCount = 0; // This will trigger efficient sequential page loading while (buffer.hasRemaining() \u0026amp;\u0026amp; lineCount \u0026lt; 1000) { byte b = buffer.get(); if (b == \u0026#39;\\n\u0026#39;) { lineCount++; } } long endTime = System.nanoTime(); System.out.println(\u0026#34;Counted \u0026#34; + lineCount + \u0026#34; lines\u0026#34;); System.out.println(\u0026#34;Sequential access took: \u0026#34; + (endTime - startTime) / 1_000_000 + \u0026#34;ms\u0026#34;); System.out.println(\u0026#34;The OS optimized this by reading ahead!\u0026#34;); } private static void demonstratePatternSearch(MappedByteBuffer buffer) { System.out.println(\u0026#34;\\n=== Pattern Search Test ===\u0026#34;); buffer.position(0); // Reset to beginning byte[] errorPattern = \u0026#34;ERROR\u0026#34;.getBytes(StandardCharsets.UTF_8); int errorCount = 0; long startTime = System.nanoTime(); // This shows how we can treat the entire file like a giant array while (buffer.hasRemaining()) { if (matchesPattern(buffer, errorPattern)) { errorCount++; // Skip past the matched pattern to avoid double-counting buffer.position(Math.min(buffer.position() + errorPattern.length, buffer.limit())); } else { buffer.position(buffer.position() + 1); } } long endTime = System.nanoTime(); System.out.println(\u0026#34;Found \u0026#34; + errorCount + \u0026#34; errors\u0026#34;); System.out.println(\u0026#34;Pattern search took: \u0026#34; + (endTime - startTime) / 1_000_000 + \u0026#34;ms\u0026#34;); System.out.println(\u0026#34;We just searched through the entire file without loading it all into heap memory!\u0026#34;); } private static boolean matchesPattern(MappedByteBuffer buffer, byte[] pattern) { if (buffer.remaining() \u0026lt; pattern.length) return false; int originalPosition = buffer.position(); // Check if the pattern matches at current position for (int i = 0; i \u0026lt; pattern.length; i++) { if (buffer.get() != pattern[i]) { buffer.position(originalPosition); // Reset position return false; } } buffer.position(originalPosition); // Reset position for caller return true; } // Let\u0026#39;s also show what happens with memory monitoring public static void demonstrateMemoryUsage(String path) { Runtime runtime = Runtime.getRuntime(); long beforeMapping = runtime.totalMemory() - runtime.freeMemory(); System.out.println(\u0026#34;Heap memory before mapping: \u0026#34; + beforeMapping / (1024 * 1024) + \u0026#34;MB\u0026#34;); try (RandomAccessFile file = new RandomAccessFile(path, \u0026#34;r\u0026#34;); FileChannel channel = file.getChannel()) { long fileSize = channel.size(); MappedByteBuffer buffer = channel.map(FileChannel.MapMode.READ_ONLY, 0, fileSize); long afterMapping = runtime.totalMemory() - runtime.freeMemory(); System.out.println(\u0026#34;Heap memory after mapping \u0026#34; + fileSize / (1024 * 1024) + \u0026#34;MB file: \u0026#34; + afterMapping / (1024 * 1024) + \u0026#34;MB\u0026#34;); System.out.println(\u0026#34;Heap memory increase: \u0026#34; + (afterMapping - beforeMapping) / 1024 + \u0026#34;KB\u0026#34;); System.out.println(\u0026#34;See? The file data isn\u0026#39;t in our heap - it\u0026#39;s managed by the OS!\u0026#34;); // Access some data to trigger page faults for (int i = 0; i \u0026lt; Math.min(1000000, buffer.limit()); i += 4096) { buffer.get(i); // Access one byte per page (4KB) } long afterAccess = runtime.totalMemory() - runtime.freeMemory(); System.out.println(\u0026#34;Heap memory after accessing data: \u0026#34; + afterAccess / (1024 * 1024) + \u0026#34;MB\u0026#34;); System.out.println(\u0026#34;Still barely any heap usage!\u0026#34;); } catch (Exception e) { System.err.println(\u0026#34;Error: \u0026#34; + e.getMessage()); } } } The Real Magic: What Makes This So Fast? 1. No Data Copying Traditional file reading copies data from disk → OS buffer → your application\u0026rsquo;s memory. Memory mapping eliminates that middle step. Your application directly accesses the OS\u0026rsquo;s file cache.\n2. Lazy Loading The file isn\u0026rsquo;t actually read until you access specific parts. If you have a 10GB file but only read the first 100MB, only that 100MB gets loaded into RAM.\n3. Shared Memory Multiple processes can map the same file and share the same physical RAM pages. If you have 5 Java processes all analyzing the same log file, they all share the same cached data.\n4. OS Optimization The operating system is incredibly smart about predicting what you\u0026rsquo;ll read next. If you\u0026rsquo;re reading sequentially, it starts loading the next few pages in advance. If you\u0026rsquo;re jumping around randomly, it keeps recently accessed pages in cache.\nThe Dark Side: When Memory Mapping Goes Wrong As we know every magic trick have its costs or its dark side let us explore what are the core costs and limitations of Memory Mapping a file.\nThe 32-bit Limitation On 32-bit systems, you can only map about 2-4GB total across your entire process. Try to map a 10GB file, and you\u0026rsquo;ll get an IOException. This is because virtual address space is limited to 4GB total, and your heap, stack, and other stuff need space too.\n1 2 3 4 5 6 // This will fail on 32-bit systems with large files try { MappedByteBuffer buffer = channel.map(FileChannel.MapMode.READ_ONLY, 0, tenGigabyteFileSize); } catch (IOException e) { System.err.println(\u0026#34;Probably hit virtual address space limit: \u0026#34; + e.getMessage()); } The File Locking Problem While your file is mapped, the OS holds a lock on it. Other processes can\u0026rsquo;t delete or modify the file. I\u0026rsquo;ve seen production deployments fail because a mapped log file couldn\u0026rsquo;t be rotated.\nThe Platform Dependency Memory mapping performance varies wildly between operating systems. Windows, Linux, and macOS all implement it differently, with different page sizes and caching strategies.\nPerformance Showdown: The Numbers Don\u0026rsquo;t Lie I ran some benchmarks on a 200MB log file to show you the difference:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 === Files.readAllLines() Test === Heap memory before reading: 3MB Lines read: 1673704 Reading took: 793ms Heap memory after reading: 311MB Heap memory increase: 308MB Notice: The entire file is now loaded in heap memory! Found 700 error lines === BufferedReader Test === Heap memory before reading: 313MB Lines read: 1673704 Reading took: 836ms Heap memory after reading: 498MB Heap memory increase: 185MB Notice: BufferedReader processes line by line without loading entire file! Found 700 error lines File size: 200MB ==== Memory Mapping Test ==== === Random Access Test === Byte at position 0: 2 Byte at position 52428859: e Byte at position 104857719: ] Byte at position 157286578: Byte at position 209715437: Random access took: 4ms Notice how fast that was? No disk seeks needed after the first few page faults! === Sequential Access Test === Counted 1000 lines Sequential access took: 4ms The OS optimized this by reading ahead! === Pattern Search Test === Found 700 errors Pattern search took: 612ms We just searched through the entire file without loading it all into heap memory! Heap memory before mapping: 500MB Heap memory after mapping 200MB file: 500MB Heap memory increase: 0KB See? The file data isn\u0026#39;t in our heap - it\u0026#39;s managed by the OS! Heap memory after accessing data: 502MB Still barely any heap usage! Process finished with exit code 0 The memory-mapped version is really faster than traditional reading and used 100x less memory in pattern search.\nWhen to Use Memory Mapping Perfect scenarios:\nLog analysis tools that need to jump around files Database implementations (this is how most databases work internally) Image processing where you need random pixel access Any time you need to treat a file like a giant array Avoid when:\nYou\u0026rsquo;re on a 32-bit system with large files The file changes frequently while you\u0026rsquo;re reading it You only need to read the file once, sequentially You\u0026rsquo;re in a memory-constrained environment where you can\u0026rsquo;t trust the OS page cache The beauty of memory mapping is that it turns file I/O into memory access, letting the operating system\u0026rsquo;s decades of optimization work for you. It\u0026rsquo;s like having a genius butler who anticipates your every need and fetches data before you even know you want it like Alfred. The full code is in my github\n","permalink":"https://mohibul.pages.dev/posts/posts/java/100daysofjava/day91/","summary":"Processing giant files - Memory mapping Very Large Files in Java Without Breaking a Sweat","title":"Day 91: The Art of Reading Massive Files Without Breaking a Sweat - Memory Mapping in Java 21"},{"content":"Ever had that sinking feeling? A critical application slows to a crawl in production. Users are complaining. You\u0026rsquo;re staring at logs, but they tell you nothing. You wish you could just ask the application, \u0026ldquo;What\u0026rsquo;s taking you so long?\u0026rdquo; Debugging is out; you can\u0026rsquo;t just halt a live system. This is where commercial giants like New Relic or Datadog wave their magic wands and produce beautiful dashboards showing you exactly which methods are the culprits.\nI\u0026rsquo;ve seen developers wrestle with these black boxes. But what if I told you that the \u0026ldquo;magic\u0026rdquo; isn\u0026rsquo;t magic at all? It\u0026rsquo;s a standard, if somewhat hidden, feature of the JVM. Today, on day 90 of your Java journey, we are going to explore the keys to this kingdom. We\u0026rsquo;re going to build our own Java Agent.\nWe\u0026rsquo;ll create a simple, powerful tool that can attach to any Java application and measure method execution times, all without touching a single line of the target application\u0026rsquo;s source code.\nBut before that let us understand what is an APM.\nWhat is an APM APM stands for Application Performance Monitoring. It\u0026rsquo;s a system that helps monitor, manage, and optimize the performance of software applications, ensuring they are available and performant for users\nNow let us move on to making a very simple Agent which can monitor the timing of the method execution. But before that let us understand what is a java agent?\nWhat is a Java Agent? Not AI but The Spy Inside Think of a Java Agent as a spy you send into the JVM. It\u0026rsquo;s a special JAR file that the JVM gives special privileges to. Its primary mission, using the java.lang.instrument API, is to intercept classes as they are being loaded. Before a class even gets a chance to run, our agent can step in and rewrite its bytecode on the fly.\nThere are two ways to deploy your agent:\nStatic Attachment: You specify the agent at startup with the -javaagent flag. This is like having your spy embedded from the very beginning.\nDynamic Attachment: You attach the agent to an already running JVM. This is the \u0026ldquo;emergency extraction\u0026rdquo; of diagnostics, more complex but incredibly powerful.\nWe\u0026rsquo;ll focus on the static approach today. It\u0026rsquo;s the perfect way to learn what is a java agent.\nThe Agent\u0026rsquo;s Entry Point: The premain Method Just as your application has a main method, our agent has a premain method. The name says it all: it runs before main. This is our beachhead. The JVM calls this method and hands us the most important tool in our arsenal: the Instrumentation object. This object is our license to modify code.\nLet\u0026rsquo;s look at the skeleton of our agent.\n1 2 3 4 5 6 7 8 9 10 11 12 // In our Agent project package com.example.javaagent; import java.lang.instrument.Instrumentation; public class MyTimingAgent { public static void premain(String agentArgs, Instrumentation inst) { System.out.println(\u0026#34;Starting our custom Java Agent...\u0026#34;); // The magic will happen here! } } For the JVM to recognize this as an agent, we need to add a special entry to our JAR\u0026rsquo;s META-INF/MANIFEST.MF file. We\u0026rsquo;ll let Maven\u0026rsquo;s maven-jar-plugin handle this configuration for us. We have another full section about what should this manifest file do and how to write it, a little bit later.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 \u0026lt;build\u0026gt; \u0026lt;plugins\u0026gt; \u0026lt;plugin\u0026gt; \u0026lt;groupId\u0026gt;org.apache.maven.plugins\u0026lt;/groupId\u0026gt; \u0026lt;artifactId\u0026gt;maven-jar-plugin\u0026lt;/artifactId\u0026gt; \u0026lt;version\u0026gt;3.3.0\u0026lt;/version\u0026gt; \u0026lt;configuration\u0026gt; \u0026lt;archive\u0026gt; \u0026lt;manifest\u0026gt; \u0026lt;addClasspath\u0026gt;true\u0026lt;/addClasspath\u0026gt; \u0026lt;/manifest\u0026gt; \u0026lt;manifestEntries\u0026gt; \u0026lt;Premain-Class\u0026gt; com.example.javaagent.MyTimingAgent \u0026lt;/Premain-Class\u0026gt; \u0026lt;/manifestEntries\u0026gt; \u0026lt;/archive\u0026gt; \u0026lt;/configuration\u0026gt; \u0026lt;/plugin\u0026gt; \u0026lt;/plugins\u0026gt; \u0026lt;/build\u0026gt; The Right Tool for the Job: Byte Buddy Now for the fun part: rewriting bytecode. We could, in theory, do this manually with a low-level library like ASM. That\u0026rsquo;s like trying to assemble a watch from raw metal—incredibly difficult and error-prone. We\u0026rsquo;re going to work smarter. We\u0026rsquo;ll use Byte Buddy, a brilliant, modern library that makes bytecode manipulation feel less like arcane magic and more like writing a fluent, readable builder.\nFirst, add the Byte Buddy dependency to your agent\u0026rsquo;s pom.xml.\n1 2 3 4 5 6 7 8 9 10 11 12 \u0026lt;dependencies\u0026gt; \u0026lt;dependency\u0026gt; \u0026lt;groupId\u0026gt;net.bytebuddy\u0026lt;/groupId\u0026gt; \u0026lt;artifactId\u0026gt;byte-buddy\u0026lt;/artifactId\u0026gt; \u0026lt;version\u0026gt;1.14.16\u0026lt;/version\u0026gt; \u0026lt;/dependency\u0026gt; \u0026lt;dependency\u0026gt; \u0026lt;groupId\u0026gt;net.bytebuddy\u0026lt;/groupId\u0026gt; \u0026lt;artifactId\u0026gt;byte-buddy-agent\u0026lt;/artifactId\u0026gt; \u0026lt;version\u0026gt;1.14.16\u0026lt;/version\u0026gt; \u0026lt;/dependency\u0026gt; \u0026lt;/dependencies\u0026gt; Our plan: Mark the Target: We\u0026rsquo;ll create a custom @Monitor annotation to mark the methods we want to time. Define the Action: We\u0026rsquo;ll create an \u0026ldquo;advice\u0026rdquo; class that holds the timing logic we want to inject. Execute the Mission: We\u0026rsquo;ll tell Byte Buddy to find all methods marked with @Monitor and wrap them with our timing logic. First, the @Monitor annotation. This can live in a separate shared JAR, but for simplicity, we\u0026rsquo;ll define it within the agent project.\n1 2 3 4 5 6 7 8 9 10 11 package com.example.javaagent.annotations; import java.lang.annotation.ElementType; import java.lang.annotation.Retention; import java.lang.annotation.RetentionPolicy; import java.lang.annotation.Target; @Retention(RetentionPolicy.RUNTIME) @Target(ElementType.METHOD) public @interface Monitor { } Next, our \u0026ldquo;advice\u0026rdquo;. This is the code that Byte Buddy will weave into the target methods. The @Advice annotations make this incredibly intuitive.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 // In our Agent project package com.example.javaagent; import net.bytebuddy.asm.Advice; public class TimerAdvice { @Advice.OnMethodEnter public static long onEnter() { // Executed at the start of the method return System.nanoTime(); } @Advice.OnMethodExit public static void onExit(@Advice.Origin String method, @Advice.Enter long startTime) { // Executed at the end of the method long duration = System.nanoTime() - startTime; System.out.printf(\u0026#34;Method \u0026#39;%s\u0026#39; executed in %d ns (%d ms)%n\u0026#34;, method, duration, duration / 1_000_000); } } Finally, we update our premain method to put Byte Buddy to work.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 // In our Agent project package com.example.javaagent; import com.example.javaagent.annotations.Monitor; import net.bytebuddy.agent.builder.AgentBuilder; import net.bytebuddy.asm.Advice; import net.bytebuddy.matcher.ElementMatchers; import java.lang.instrument.Instrumentation; public class MyTimingAgent { public static void premain(String agentArgs, Instrumentation inst) { System.out.println(\u0026#34;Starting our custom Java Agent...\u0026#34;); new AgentBuilder.Default() .type(ElementMatchers.any()) // Look at any class .transform((builder, typeDescription, classLoader, module, protectionDomain) -\u0026gt; builder.method(ElementMatchers.isAnnotatedWith(Monitor.class)) // Find methods with our annotation .intercept(Advice.to(TimerAdvice.class)) // Apply our advice ).installOn(inst); System.out.println(\u0026#34;Agent installation complete.\u0026#34;); } } This beautiful, fluent code is our mission directive. It tells Byte Buddy to find any method, anywhere, that has our @Monitor annotation and wrap it with the logic from TimerAdvice.\nAttaching Agent To The Target Application Now, let\u0026rsquo;s create a simple application to test our agent on. This is a completely separate project.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 // In our Target Application project import com.example.javaagent.annotations.Monitor; public class MonitoredApp { @Monitor public void doWork() throws InterruptedException { System.out.println(\u0026#34;--\u0026gt; Doing some important work...\u0026#34;); Thread.sleep(200); System.out.println(\u0026#34;--\u0026gt; Work finished.\u0026#34;); } @Monitor public void doMoreWork() throws InterruptedException { System.out.println(\u0026#34;--\u0026gt; Doing even more work...\u0026#34;); Thread.sleep(500); System.out.println(\u0026#34;--\u0026gt; More work finished.\u0026#34;); } public void doUnmonitoredWork() { System.out.println(\u0026#34;--\u0026gt; This work is not being monitored.\u0026#34;); } public static void main(String[] args) throws InterruptedException { MonitoredApp app = new MonitoredApp(); app.doWork(); app.doMoreWork(); app.doUnmonitoredWork(); } } Notice how clean this is. The application code only knows about the @Monitor annotation. It has no idea its methods are about to be timed by an external agent.\nThe Grand Finale: Running with the Agent Build the Agent: In your agent project, run mvn clean package. This creates your agent JAR (e.g., agent-1.0.jar). Build the App: Compile your MonitoredApp. Launch: Run the application using the -javaagent flag. The Agent Manifest: Your License to Spy The agent should have a manifest file where it is described. The manifest file (META-INF/MANIFEST.MF) is what tells the JVM how to handle your agent JAR. Here\u0026rsquo;s what it should contain:\nManifest-Version: 1.0\rPremain-Class: com.example.javaagent.MyTimingAgent\rAgent-Class: com.example.javaagent.MyTimingAgent\rCan-Redefine-Classes: true\rCan-Retransform-Classes: true\rBoot-Classpath-Append: byte-buddy-1.14.16.jar Key entries explained:\nPremain-Class: The class containing the premain method for static attachment Agent-Class: The class containing the agentmain method for dynamic attachment (optional for our example) Can-Redefine-Classes: Allows the agent to redefine classes Can-Retransform-Classes: Allows the agent to retransform classes that have already been loaded Boot-Classpath-Append: Adds required dependencies to the bootstrap classpath Maven will automatically generate this manifest when you build with the configuration we showed earlier in the pom.xml.\n1 2 3 4 5 6 # Assuming agent JAR is in agent/target/ and app classes are in app/target/classes/ # Make sure the annotation JAR is also on the classpath if it\u0026#39;s separate java -javaagent:path/to/my-agent-1.0.jar \\ -cp path/to/target/application/classes:path/to/annotations.jar \\ MonitoredApp When you hit enter, you\u0026rsquo;ll see the magic happen.\nAgent deployed. Starting transformation...\rTransformation complete. Agent is active.\r--\u0026gt; Doing some important work...\r--\u0026gt; Work finished.\rMethod \u0026#39;public void MonitoredApp.doWork() throws java.lang.InterruptedException\u0026#39; took 202 ms\r--\u0026gt; Doing even more work...\r--\u0026gt; More work finished.\rMethod \u0026#39;public void MonitoredApp.doMoreWork() throws java.lang.InterruptedException\u0026#39; took 503 ms\r--\u0026gt; This work is not being monitored.\r---\r... There it is. Our agent successfully intercepted the annotated methods and reported their execution time, proving that the \u0026ldquo;magic\u0026rdquo; of APM is well within your grasp. You\u0026rsquo;ve just built a foundational tool that professional engineers use to keep complex systems running smoothly.\nWelcome to the world of Java instrumentation. The spy is in.\n","permalink":"https://mohibul.pages.dev/posts/posts/java/100daysofjava/day90/","summary":"The Spy Inside Your JVM - Creating Java Agents That Monitor your apps","title":"Day 90: How I Built a Secret Agent That Infiltrates Any Java Application"},{"content":"The Story Begins: When One Becomes Many Imagine: You\u0026rsquo;re building a bank\u0026rsquo;s core system, and you need a logger that records every transaction. In your ideal world, this logger should be like that one reliable friend everyone turns to and there\u0026rsquo;s only one person and everyone knows where to find them.\nYou craft what seems like the perfect solution:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 public class TransactionLogger { private static TransactionLogger instance; private TransactionLogger() { // Initialize expensive logging resources System.out.println(\u0026#34;Logger initialized at: \u0026#34; + System.currentTimeMillis()); } public static TransactionLogger getInstance() { if (instance == null) { instance = new TransactionLogger(); } return instance; } public void log(String transaction) { System.out.println(\u0026#34;LOG: \u0026#34; + transaction); } } This looks elegant, doesn\u0026rsquo;t it? One class, one instance, problem solved. But there\u0026rsquo;s a twist in our story—and it\u0026rsquo;s about to get interesting.\nWhen Reality Strikes: The Race Condition Drama Imagine our bank system goes live. Hundreds of customers are making transactions simultaneously. Multiple threads are calling getInstance() at the exact same moment. What happens next is like a scene from a comedy of errors:\nThread A: \u0026ldquo;Is instance null? Yes! I\u0026rsquo;ll create one.\u0026rdquo; Thread B: \u0026ldquo;Is instance null? Yes! I\u0026rsquo;ll create one too.\u0026rdquo; Thread C: \u0026ldquo;Is instance null? Still yes! Another one coming up.\u0026rdquo; Suddenly, our \u0026ldquo;one and only\u0026rdquo; logger has siblings—multiple instances running around, each thinking they\u0026rsquo;re the chosen one. Our singleton has become a \u0026ldquo;multiple-ton,\u0026rdquo; and chaos ensues.\nLet\u0026rsquo;s see this story unfold into code:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 @Test void demonstrateTheChaoticScenario() throws InterruptedException { int numberOfThreads = 100; Set\u0026lt;TransactionLogger\u0026gt; instances = ConcurrentHashMap.newKeySet(); CountDownLatch latch = new CountDownLatch(numberOfThreads); // Launch a hundred threads simultaneously for (int i = 0; i \u0026lt; numberOfThreads; i++) { new Thread(() -\u0026gt; { instances.add(TransactionLogger.getInstance()); latch.countDown(); }).start(); } latch.await(); // The shocking revelation System.out.println(\u0026#34;Expected: 1 instance, Actually got: \u0026#34; + instances.size()); // You might see: \u0026#34;Expected: 1 instance, Actually got: 7\u0026#34; (or any number \u0026gt; 1) } The Heroes Enter: Thread-Safe Patterns to the Rescue Now that we\u0026rsquo;ve seen the villain (race conditions) in action, let\u0026rsquo;s meet our heroes, various thread-safe singleton patterns, each with their own superpowers and weaknesses.\nHero #1: The Traffic Controller (Synchronized Method) Our first hero is like a traffic controller at a busy intersection—only one car gets the green light at a time.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 public class SynchronizedTransactionLogger { private static SynchronizedTransactionLogger instance; private SynchronizedTransactionLogger() { System.out.println(\u0026#34;Synchronized logger created by: \u0026#34; + Thread.currentThread().getName()); } public static synchronized SynchronizedTransactionLogger getInstance() { if (instance == null) { instance = new SynchronizedTransactionLogger(); } return instance; } } The Good: Simple and foolproof. No matter how many threads arrive at the intersection, only one gets to proceed at a time.\nThe Not-So-Good: Here\u0026rsquo;s the catch—even after the construction work at the intersection is done and traffic should flow freely, our traffic controller keeps stopping every single car to check if construction is still ongoing. Every call to getInstance() waits for the red light, even when it\u0026rsquo;s completely unnecessary.\nThink of it like this: after the first car creates the instance, every subsequent call still has to wait at the traffic light, like being stuck in a construction zone where the work finished hours ago but they forgot to remove the temporary traffic control.\nHero #2: The Eager Beaver (Eager Initialization) Our second hero doesn\u0026rsquo;t wait for anyone. They show up to the party early and stay till the end.\n1 2 3 4 5 6 7 8 9 10 11 12 public class EagerTransactionLogger { // Created immediately when class is loaded private static final EagerTransactionLogger instance = new EagerTransactionLogger(); private EagerTransactionLogger() { System.out.println(\u0026#34;Eager logger ready before anyone asked!\u0026#34;); } public static EagerTransactionLogger getInstance() { return instance; // No checks needed, already there! } } The Good: Lightning fast access and completely thread-safe. The JVM\u0026rsquo;s class loader mechanism guarantees that static fields are initialized exactly once, before any thread can access the class.\nThe Not-So-Good: Like that friend who always arrives two hours early to every party, even the ones they might not attend. If your singleton is resource-heavy and might never be used, you\u0026rsquo;re paying the cost upfront whether you need it or not.\nHero #3: The Double Agent (Double-Checked Locking) Our third hero is sophisticated—they have a two-step verification process that\u0026rsquo;s both secure and efficient.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 public class DoubleCheckedTransactionLogger { // The volatile keyword is crucial here! private static volatile DoubleCheckedTransactionLogger instance; private DoubleCheckedTransactionLogger() { System.out.println(\u0026#34;Double-checked logger created after careful verification\u0026#34;); } public static DoubleCheckedTransactionLogger getInstance() { // First check - no synchronization needed if already created if (instance == null) { synchronized (DoubleCheckedTransactionLogger.class) { // Second check - inside synchronized block if (instance == null) { instance = new DoubleCheckedTransactionLogger(); } } } return instance; } } This pattern is like a smart security system:\nFirst Guard: \u0026ldquo;Is there already someone inside? Yes? Go ahead!\u0026rdquo; Second Guard: \u0026ldquo;Wait, let me double-check inside this secure room. Nope, still empty? Okay, you can enter.\u0026rdquo; The Good: After initialization, accessing the instance is as fast as the eager approach. The first null check eliminates synchronization overhead for subsequent calls.\nThe Critical Detail: The volatile keyword isn\u0026rsquo;t just decoration—it\u0026rsquo;s essential! Without it, you might see a partially constructed object due to the way modern processors reorder instructions. It\u0026rsquo;s like ensuring everyone sees the same version of reality at the same time.\nHero #4: The Hidden Master (Initialization-on-Demand Holder) Our fourth hero uses ancient Java wisdom—the power of class loading mechanics.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 public class HolderTransactionLogger { private HolderTransactionLogger() { System.out.println(\u0026#34;Holder logger created by the master of lazy loading\u0026#34;); } // The secret weapon: a static inner class private static class LoggerHolder { private static final HolderTransactionLogger INSTANCE = new HolderTransactionLogger(); } public static HolderTransactionLogger getInstance() { return LoggerHolder.INSTANCE; // Class loading magic happens here } } This is like having a wise mentor who only appears when you truly need them. The LoggerHolder class isn\u0026rsquo;t loaded until someone calls getInstance(), and when it is loaded, the JVM guarantees thread-safe initialization.\nThe Good: Lazy initialization without any explicit synchronization. It\u0026rsquo;s elegant, fast, and leverages the JVM\u0026rsquo;s built-in thread safety guarantees.\nThe Beautiful Part: No locks, no volatile keywords, no complex logic—just pure Java class loading mechanics doing the heavy lifting.\nHero #5: The Royal Guard (Enum Singleton) Our final hero is royalty—they have built-in protection against all forms of attack.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 public enum EnumTransactionLogger { INSTANCE; private EnumTransactionLogger() { System.out.println(\u0026#34;Enum logger: Born to be unique!\u0026#34;); } public void log(String transaction) { System.out.println(\u0026#34;ROYAL LOG: \u0026#34; + transaction); } public void doSomething() { System.out.println(\u0026#34;Performing singleton operations...\u0026#34;); } } // Usage is simple and elegant EnumTransactionLogger.INSTANCE.log(\u0026#34;Transfer completed\u0026#34;); The Good:\nThread-safe by design Immune to reflection attacks (try to break it, you can\u0026rsquo;t!) Handles serialization correctly (no multiple instances even after deserialization) Simple and clean syntax The Limitation: Enums can\u0026rsquo;t extend other classes, which might limit your design flexibility in some scenarios.\nThe Ultimate Showdown: Choosing Your Champion Let me tell you a story about choosing the right tool for the job. Imagine you\u0026rsquo;re a chef, and each singleton pattern is a different kitchen appliance:\nPattern When to Use Like Choosing\u0026hellip; Synchronized Method Simple applications, low concurrency A reliable old mixer - slow but dependable Eager Initialization Always-needed singletons, startup cost acceptable A rice cooker - ready when you walk in Double-Checked Locking High performance needed, comfortable with complexity A professional espresso machine - complex but perfect results Holder Pattern Most general cases A Swiss Army knife - versatile and elegant Enum Singleton Maximum protection needed A bank vault - unbreakable security Testing Our Heroes: Proving They Work Let\u0026rsquo;s put our patterns through their paces with a stress test that would make any singleton proud:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 @Test void stressTestAllSingletonPatterns() throws InterruptedException { testSingletonThreadSafety(\u0026#34;Holder Pattern\u0026#34;, HolderTransactionLogger::getInstance); testSingletonThreadSafety(\u0026#34;Enum Pattern\u0026#34;, () -\u0026gt; EnumTransactionLogger.INSTANCE); testSingletonThreadSafety(\u0026#34;Synchronized\u0026#34;, SynchronizedTransactionLogger::getInstance); testSingletonThreadSafety(\u0026#34;Double-Checked\u0026#34;, DoubleCheckedTransactionLogger::getInstance); testSingletonThreadSafety(\u0026#34;Eager\u0026#34;, EagerTransactionLogger::getInstance); } private \u0026lt;T\u0026gt; void testSingletonThreadSafety(String patternName, Supplier\u0026lt;T\u0026gt; instanceSupplier) throws InterruptedException { int threadCount = 1000; Set\u0026lt;T\u0026gt; instances = ConcurrentHashMap.newKeySet(); CountDownLatch latch = new CountDownLatch(threadCount); for (int i = 0; i \u0026lt; threadCount; i++) { new Thread(() -\u0026gt; { instances.add(instanceSupplier.get()); latch.countDown(); }).start(); } latch.await(); System.out.println(patternName + \u0026#34; - Instances created: \u0026#34; + instances.size()); assertEquals(1, instances.size(), patternName + \u0026#34; should create only one instance\u0026#34;); } The Moral of Our Story In our journey through the land of thread-safe singletons, we\u0026rsquo;ve learned that the simple question \u0026ldquo;How do I create one instance?\u0026rdquo; has surprisingly complex answers in a multithreaded world.\nThe Wisdom I\u0026rsquo;ve Gained:\nThe naive approach fails - Race conditions are real and sneaky Every solution has trade-offs - Performance vs. simplicity vs. features The Holder Pattern is often the hero - Elegant, fast, and thread-safe Enum singletons are the ultimate protection - When security matters most Context matters - Choose based on your specific needs A Final Thought:\nSingletons are like that one friend everyone relies on—powerful when used correctly, but they can make your code harder to test and maintain. Sometimes, dependency injection frameworks offer better alternatives by managing object lifecycles for you.\nRemember, the best singleton pattern is often the one you don\u0026rsquo;t need to write yourself. But when you do need one, now you have an arsenal of thread-safe techniques to choose from.\nThe next time someone asks you about thread-safe singletons, you won\u0026rsquo;t just give them code—you\u0026rsquo;ll tell them a story about race conditions, heroes, and the eternal quest for that perfect \u0026ldquo;one and only\u0026rdquo; instance in the chaotic world of multithreaded programming.\nThe Implementation Showcase Here\u0026rsquo;s a complete working example that demonstrates all patterns in action:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 public class SingletonShowcase { public static void main(String[] args) throws InterruptedException { System.out.println(\u0026#34;=== Singleton Pattern Showcase ===\u0026#34;); // Test each pattern demonstrateHolderPattern(); demonstrateEnumPattern(); demonstrateDoubleCheckedLocking(); System.out.println(\u0026#34;\\n=== Thread Safety Test ===\u0026#34;); performConcurrencyTest(); } private static void demonstrateHolderPattern() { System.out.println(\u0026#34;\\n--- Holder Pattern ---\u0026#34;); HolderTransactionLogger logger1 = HolderTransactionLogger.getInstance(); HolderTransactionLogger logger2 = HolderTransactionLogger.getInstance(); System.out.println(\u0026#34;Same instance? \u0026#34; + (logger1 == logger2)); } private static void demonstrateEnumPattern() { System.out.println(\u0026#34;\\n--- Enum Pattern ---\u0026#34;); EnumTransactionLogger.INSTANCE.log(\u0026#34;Testing enum singleton\u0026#34;); System.out.println(\u0026#34;Enum instances are always the same: \u0026#34; + (EnumTransactionLogger.INSTANCE == EnumTransactionLogger.INSTANCE)); } private static void demonstrateDoubleCheckedLocking() { System.out.println(\u0026#34;\\n--- Double-Checked Locking ---\u0026#34;); DoubleCheckedTransactionLogger logger1 = DoubleCheckedTransactionLogger.getInstance(); DoubleCheckedTransactionLogger logger2 = DoubleCheckedTransactionLogger.getInstance(); System.out.println(\u0026#34;Same instance? \u0026#34; + (logger1 == logger2)); } private static void performConcurrencyTest() throws InterruptedException { int threadCount = 100; CountDownLatch latch = new CountDownLatch(threadCount); Set\u0026lt;HolderTransactionLogger\u0026gt; instances = ConcurrentHashMap.newKeySet(); for (int i = 0; i \u0026lt; threadCount; i++) { new Thread(() -\u0026gt; { instances.add(HolderTransactionLogger.getInstance()); latch.countDown(); }).start(); } latch.await(); System.out.println(\u0026#34;Concurrency test result - Unique instances: \u0026#34; + instances.size()); System.out.println(\u0026#34;Thread safety: \u0026#34; + (instances.size() == 1 ? \u0026#34;PASSED\u0026#34; : \u0026#34;FAILED\u0026#34;)); } } And there you have it—your complete guide to thread-safe singletons, served with a side of storytelling and practical wisdom. May your instances be single, your threads be safe, and your code be elegant!\n","permalink":"https://mohibul.pages.dev/posts/posts/java/100daysofjava/day89/","summary":"That perfect Singleton you wrote? It\u0026rsquo;s probably creating multiple instances in production—here\u0026rsquo;s why it happens and five bulletproof ways to fix it.","title":"Day 89: When Your Singleton Isn't Really a Singleton (And How to Fix It)"},{"content":"I worked on a production workflow that turns call recordings into structured transcripts. The workflow needed speaker labels, six languages, consistent number and date formatting, and a way to flag uncertain sections for review.\nThe implementation uses Spring Boot, Google Cloud Storage, Vertex AI, and Gemini 1.5. This article focuses on the decisions I can discuss publicly: prompt structure, model configuration, confidence signals, single and batch processing, and failure handling.\nRequirements beyond speech-to-text The workflow had five requirements:\nMultilingual Support: The system had to handle English plus five other languages without missing a beat. Speaker Diarization: We needed clear labels showing who was talking when (e.g., \u0026ldquo;Speaker A:\u0026rdquo;, \u0026ldquo;Speaker B:\u0026rdquo;), which is surprisingly hard to get right. Specific Formatting: Each transcript needed timestamps at speaker changes and language-appropriate number/date formats. Handling Imperfections: Real-world audio has issues - background noise, people talking over each other, mumbling. The system needed to handle these gracefully. Quality Assessment: We needed some way to flag transcriptions that might need human review. Architecture: A Spring Boot \u0026amp; Google Cloud Symphony I built a workflow that looks something like this:\nAudio Ingestion \u0026amp; Preparation: The backend receives audio files and converts them to FLAC format. I chose FLAC because it preserves audio quality while keeping file sizes manageable. Cloud Storage: These FLAC files get uploaded to Google Cloud Storage. This step is crucial - it lets Gemini access potentially large audio files without timing out or hitting memory limits. Vertex AI \u0026amp; Gemini 1.5: The Spring Boot application calls Vertex AI and provides the audio file\u0026rsquo;s location. The prompt carries the transcription and formatting requirements described below. Processing \u0026amp; Storage: Once Gemini does its thing, we parse the response, add our own confidence scoring, and store everything in our database. Notification/Feedback: For user-triggered transcriptions, we send back success/failure notifications. Using GCS as the middleman was a bit of extra work, but it paid off by making the system more robust when handling larger files.\nHarnessing Gemini 1.5 Gemini 1.5 supported the required audio input and context through GCS URIs, which made it suitable for this workflow.\nPrompt Engineering for Precision This was the trickiest part of the whole project. Just saying \u0026ldquo;hey, transcribe this\u0026rdquo; wasn\u0026rsquo;t going to cut it. After many iterations, I landed on this prompt structure:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 Please accurately transcribe this audio recording with high attention to detail. %1$s is the preferred language if specified. I need a professional-quality transcript that captures the following elements: Language Detection and Handling: 1. If language code is provided ([Language code]): - Use this as the primary transcription language 2. If no language code is provided: - Detect the spoken language from the audio - Transcribe in the detected language - Include the detected language code in the response Formatting Requirements: - Include timestamps at every speaker change - Label speakers as \u0026#34;Speaker A:\u0026#34;, \u0026#34;Speaker B:\u0026#34;, etc. - Use appropriate number formatting for the detected/specified language - Format dates as DD.MM.YYYY Language-Specific Guidelines: [Language code] Special Instructions: - Mark unclear speech with [inaudible] - For overlapping speech, prioritize the main speaker - Ignore background noise unless relevant This prompt spells out exactly what we need:\nThe target language (dynamically inserted at runtime) How to format timestamps, speaker labels, numbers, and dates Clear instructions for handling tricky situations like unclear speech Navigating Safety Settings I hit an unexpected roadblock with Vertex AI\u0026rsquo;s default safety settings. They\u0026rsquo;re designed to prevent harmful content generation, but in our case, they sometimes flagged legitimate business conversations containing certain keywords.\nAfter some trial and error, I identified which safety categories we could safely disable for our specific use case. This required careful consideration - we needed the full, unfiltered transcription, but we were working with known audio sources in a controlled environment.\nMeasuring Confidence: A Pragmatic Approach One challenge with LLMs is they don\u0026rsquo;t give you a simple \u0026ldquo;I\u0026rsquo;m 87% confident in this transcription\u0026rdquo; score. However, Gemini does return log probabilities (avg_logprobs) for the generated tokens, which provide some insight into the model\u0026rsquo;s internal confidence.\nI cobbled together a basic confidence scoring system:\nExtract the avg_logprobs from the API response Set a threshold based on experimentation (around -0.4) If the average log probability falls below this threshold, flag that transcript for human review It\u0026rsquo;s not perfect, but it gave us a practical way to identify potentially problematic transcriptions that might need a second pair of eyes.\nImplementation: Async vs. Batch Processing To handle different use cases, I implemented two processing modes:\nAsynchronous Processing: For user-triggered transcriptions, we process things in the background using Spring\u0026rsquo;s @Async annotations. The user gets an immediate acknowledgment, and we notify them when the job completes.\nBatch Processing: For processing backlogs of audio files, I built a scheduled job using Spring\u0026rsquo;s @Scheduled annotation. This runs during off-hours, picking up unprocessed files and working through them methodically.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 public static List\u0026lt;SafetySetting\u0026gt; getCustomSafetySettings() { final List\u0026lt;SafetySetting\u0026gt; safetySettings = new ArrayList\u0026lt;\u0026gt;(); safetySettings.add(SafetySetting.newBuilder() .setCategory(HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT) .setThreshold(SafetySetting.HarmBlockThreshold.BLOCK_NONE) .build()); safetySettings.add(SafetySetting.newBuilder() .setCategory(HarmCategory.HARM_CATEGORY_HARASSMENT) .setThreshold(SafetySetting.HarmBlockThreshold.BLOCK_NONE) .build()); safetySettings.add(SafetySetting.newBuilder() .setCategory(HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT) .setThreshold(SafetySetting.HarmBlockThreshold.BLOCK_NONE) .build()); safetySettings.add(SafetySetting.newBuilder() .setCategory(HarmCategory.HARM_CATEGORY_HATE_SPEECH) .setThreshold(SafetySetting.HarmBlockThreshold.BLOCK_NONE) .build()); safetySettings.add(SafetySetting.newBuilder() .setCategory(HarmCategory.HARM_CATEGORY_UNSPECIFIED) .setThreshold(SafetySetting.HarmBlockThreshold.BLOCK_NONE) .build()); return safetySettings; } private SpeechToTextInfo transcribeAudioWithGeminiSync(final String gcsUri, final SpeechToTextInfo pendingInfo) { try { log.debug(\u0026#34;Transcribing audio synchronously with Gemini for callId: {} and recordingId: {}\u0026#34;, pendingInfo.getCallId(), pendingInfo.getRecordId()); final GenerativeModel model = new GenerativeModel(modelName, vertexAI); model.withSafetySettings(getCustomSafetySettings()); final GenerateContentResponse response = model.generateContent( ContentMaker.fromMultiModalData( buildPrompt(\u0026#34;\u0026#34;), PartMaker.fromMimeTypeAndData(\u0026#34;audio/flac\u0026#34;, gcsUri) )); final ImmutableGeminiSpeechToTextTranscriptInfo.Builder transcriptInfoBuilder = ImmutableGeminiSpeechToTextTranscriptInfo.builder(); final double avgLogProbs = response.getCandidates(0).getAvgLogprobs(); transcriptInfoBuilder.hasAcceptableConfidence(avgLogProbs \u0026gt;= CONFIDENCE_THRESHOLD) .avgLogProbs(avgLogProbs); final String transcript = ResponseHandler.getText(response); final SpeechToTextInfo completedInfo = SpeechToTextInfo.from(pendingInfo) .setTranscript(transcript) .setStatus(SpeechToTextState.success) .setTranscriptsInfo(ImmutableList.of(transcriptInfoBuilder.build())) .build(); // save into db return completedInfo; } catch (final Exception e) { log.error(\u0026#34;Failed to transcribe audio synchronously for callId: {}\u0026#34;, pendingInfo.getCallId(), e); final SpeechToTextInfo failedInfo = SpeechToTextInfo.from(pendingInfo) .setStatus(SpeechToTextState.failed) .build(); // save into db return failedInfo; } } private void transcribeAudioWithGemini(final String gcsUri, final String language, final SpeechToTextInfo pendingInfo) { try { log.debug(\u0026#34;Transcribing audio with Gemini for callId: {} and recordingId: {}\u0026#34;, pendingInfo.getCallId(), pendingInfo.getRecordId()); final GenerativeModel model = new GenerativeModel(modelName, vertexAI); model.withSafetySettings(getCustomSafetySettings()); final ApiFuture\u0026lt;GenerateContentResponse\u0026gt; response = model.generateContentAsync( ContentMaker.fromMultiModalData( buildPrompt(language), PartMaker.fromMimeTypeAndData(\u0026#34;audio/flac\u0026#34;, gcsUri) )); log.debug(\u0026#34;Transcribing audio with Gemini for callId: {} and recordingId: {} and response {}\u0026#34;, pendingInfo.getCallId(), pendingInfo.getRecordId(), response); ApiFutures.addCallback(response, new ApiFutureCallback\u0026lt;GenerateContentResponse\u0026gt;() { @Override public void onSuccess(final GenerateContentResponse result) { log.debug(\u0026#34;Transcribing audio with Gemini for callId: {} and recordingId: {} and result {}\u0026#34;, pendingInfo.getCallId(), pendingInfo.getRecordId(), result); processTranscription(result, pendingInfo); } @Override public void onFailure(final Throwable t) { handleTranscriptionFailure(pendingInfo, t); } }, MoreExecutors.directExecutor()); } catch (final Exception e) { handleTranscriptionFailure(pendingInfo, e); } } What I learned Audio prompts need explicit output rules. Small wording changes can alter speaker labels, timestamps, and number formatting. Safety settings are part of the product behavior and need tests that reflect the audio domain. avg_logprobs is not a calibrated confidence score. It can still help route likely failures for review when the threshold is evaluated against known examples. Single-item and batch flows need the same lifecycle and failure semantics, even when their execution paths differ. Limitations This article omits customer data, private evaluation sets, traffic volume, and provider reliability measurements. The examples explain the architecture and code patterns, but they do not provide a public accuracy benchmark for Gemini 1.5 or compare it with every transcription provider.\n","permalink":"https://mohibul.pages.dev/posts/posts/java/100daysofjava/day88/","summary":"Architecture and implementation notes for multilingual audio transcription with Spring Boot, Google Cloud Storage, Vertex AI, confidence signals, and batch processing.","title":"Building production audio transcription with Gemini 1.5 and Spring Boot"},{"content":"Have you ever needed to access or modify environment variables in your Java application? If you\u0026rsquo;re like me, you\u0026rsquo;ve probably reached for System.getenv() to read variables, but then scratched your head wondering how to update them.\nIn today\u0026rsquo;s post, I\u0026rsquo;ll dive into environment variables in Java 21 - something I\u0026rsquo;ve been playing with recently. We\u0026rsquo;ll look at how to read them (the easy part) and explore different approaches to modify them (the not-so-easy part).\nTLDR Java makes reading env variables easy (System.getenv()), but modifying them is restricted\nFour options for modification: ProcessBuilder for child processes only JNA for native system function access Reflection hacks (avoid in production!) Native OS commands (setx/export) Best practice: Load env variables at startup rather than modifying at runtime What are Environment Variables anyway? Before jumping into code, let\u0026rsquo;s take a step back. Environment variables are basically key-value pairs that live outside your application. They\u0026rsquo;re part of the environment in which your process runs.\nI like to think of them as a global configuration that follows your application around. They\u0026rsquo;re incredibly useful for things like:\nStoring database connection strings (so they\u0026rsquo;re not hardcoded in your app) Setting up API keys (without exposing them in your code repository) Configuring different behavior across environments (dev/staging/prod) Storing system paths and user information If you\u0026rsquo;re working with containerized applications or microservices, you\u0026rsquo;re probably already using environment variables extensively.\nReading Environment Variables in Java Let\u0026rsquo;s start with the easy part - reading environment variables. Java provides a couple of straightforward methods for this.\nThe Classic System.getenv() Approach This is the method I use :\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 import java.util.Map; public class EnvReader { public static void main(String[] args) { // Get a specific variable String path = System.getenv(\u0026#34;PATH\u0026#34;); System.out.println(\u0026#34;My PATH is: \u0026#34; + path); // Or get all environment variables at once Map\u0026lt;String, String\u0026gt; env = System.getenv(); System.out.println(\u0026#34;All my environment variables:\u0026#34;); env.forEach((key, value) -\u0026gt; System.out.println(key + \u0026#34;: \u0026#34; + value)); } } Simple and effective! You can either get a specific variable by name or grab all of them as a Map.\nUsing ProcessBuilder for Environment Access There\u0026rsquo;s another approach using ProcessBuilder that I find useful, especially when working with subprocesses:\n1 2 3 4 5 6 7 8 9 10 11 import java.util.Map; public class ProcessBuilderEnvReader { public static void main(String[] args) { ProcessBuilder processBuilder = new ProcessBuilder(); Map\u0026lt;String, String\u0026gt; environment = processBuilder.environment(); System.out.println(\u0026#34;My environment variables via ProcessBuilder:\u0026#34;); environment.forEach((key, value) -\u0026gt; System.out.println(key + \u0026#34; = \u0026#34; + value)); } } The nice thing about ProcessBuilder is that you can modify this environment map for any subprocesses you launch - more on that shortly.\nThe Tricky Part: Modifying Environment Variables Now comes the challenging part. Unlike reading environment variables, Java doesn\u0026rsquo;t provide direct methods to modify them in the current process. This is actually by design for security reasons.\nHowever, there are several workarounds we can use. Let\u0026rsquo;s explore them.\nApproach 1: Modifying Environment for Child Processes This is the most straightforward approach. While you can\u0026rsquo;t change the current process\u0026rsquo;s environment, you can set environment variables for any child processes you launch:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 import java.io.IOException; public class EnvSetterForChildProcess { public static void main(String[] args) { try { ProcessBuilder pb = new ProcessBuilder(\u0026#34;java\u0026#34;, \u0026#34;-version\u0026#34;); // Here\u0026#39;s where the magic happens pb.environment().put(\u0026#34;MY_CUSTOM_VAR\u0026#34;, \u0026#34;Hello from parent!\u0026#34;); Process process = pb.start(); int exitCode = process.waitFor(); System.out.println(\u0026#34;Process finished with exit code: \u0026#34; + exitCode); } catch (IOException | InterruptedException e) { e.printStackTrace(); } } } The limitation? This only affects the child process, not your current JVM or the system as a whole.\nApproach 2: Using JNA (Java Native Access) For more control, sometimes we should look into JNA, which lets you access native system functions:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 import com.sun.jna.Library; import com.sun.jna.Native; import com.sun.jna.Platform; public class JNAEnvSetter { // Interface to C library functions public interface CLibrary extends Library { CLibrary INSTANCE = Native.load(Platform.isWindows() ? \u0026#34;msvcrt\u0026#34; : \u0026#34;c\u0026#34;, CLibrary.class); int setenv(String name, String value, int overwrite); int unsetenv(String name); } public static void main(String[] args) { // Set an environment variable int result = CLibrary.INSTANCE.setenv(\u0026#34;MY_ENV_VAR\u0026#34;, \u0026#34;my_value\u0026#34;, 1); if (result == 0) { System.out.println(\u0026#34;Environment variable set successfully\u0026#34;); // Verify by reading it back String value = System.getenv(\u0026#34;MY_ENV_VAR\u0026#34;); System.out.println(\u0026#34;MY_ENV_VAR: \u0026#34; + value); } else { System.out.println(\u0026#34;Failed to set environment variable\u0026#34;); } } } Remember to add JNA to your project dependencies if you want to try this approach.\nApproach 3: The Reflection Hack (Use with Caution!) I\u0026rsquo;ve seen this approach used, but I generally don\u0026rsquo;t recommend it for production code. It uses reflection to modify the private environment map:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 import java.lang.reflect.*; import java.util.Map; public class ReflectionEnvSetter { @SuppressWarnings(\u0026#34;unchecked\u0026#34;) public static void main(String[] args) { try { // Attempt to update environment variables via ProcessEnvironment fields. Class\u0026lt;?\u0026gt; processEnvironmentClass = Class.forName(\u0026#34;java.lang.ProcessEnvironment\u0026#34;); Field envField = processEnvironmentClass.getDeclaredField(\u0026#34;theEnvironment\u0026#34;); envField.setAccessible(true); Map\u0026lt;String, String\u0026gt; env = (Map\u0026lt;String, String\u0026gt;) envField.get(null); env.putAll(newEnv); // Try to update the case-insensitive environment as well, if present. try { Field ciEnvField = processEnvironmentClass.getDeclaredField(\u0026#34;theCaseInsensitiveEnvironment\u0026#34;); ciEnvField.setAccessible(true); @SuppressWarnings(\u0026#34;unchecked\u0026#34;) Map\u0026lt;String, String\u0026gt; ciEnv = (Map\u0026lt;String, String\u0026gt;) ciEnvField.get(null); ciEnv.putAll(newEnv); } catch (NoSuchFieldException nsfe) { // Field not present on this system (e.g., non-Windows), ignore. } } catch (ClassNotFoundException | NoSuchFieldException | IllegalAccessException e) { // Fallback: Modify the unmodifiable map returned by System.getenv() try { Map\u0026lt;String, String\u0026gt; sysEnv = System.getenv(); // Use reflection to access the internal modifiable map Field field = sysEnv.getClass().getDeclaredField(\u0026#34;m\u0026#34;); field.setAccessible(true); Map\u0026lt;String, String\u0026gt; modifiableEnv = (Map\u0026lt;String, String\u0026gt;) field.get(sysEnv); modifiableEnv.putAll(newEnv); } catch (Exception ex) { throw new RuntimeException(\u0026#34;Failed to set environment variables\u0026#34;, ex); } } } } Why don\u0026rsquo;t I recommend this? It\u0026rsquo;s:\nNot guaranteed to work across JVM implementations May behave differently in future Java versions May lead to unexpected behavior or security issues I\u0026rsquo;ve seen it not work properly so proceed with caution!\nApproach 4: Native OS Commands Sometimes the simplest approach is to just execute the appropriate OS command:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 import java.io.IOException; public class NativeEnvSetter { public static void main(String[] args) { try { ProcessBuilder processBuilder; if (System.getProperty(\u0026#34;os.name\u0026#34;).toLowerCase().contains(\u0026#34;windows\u0026#34;)) { // Windows way processBuilder = new ProcessBuilder(\u0026#34;cmd\u0026#34;, \u0026#34;/c\u0026#34;, \u0026#34;setx\u0026#34;, \u0026#34;MY_ENV_VAR\u0026#34;, \u0026#34;my_value\u0026#34;); } else { // Unix/Linux/Mac way processBuilder = new ProcessBuilder(\u0026#34;/bin/sh\u0026#34;, \u0026#34;-c\u0026#34;, \u0026#34;export MY_ENV_VAR=my_value\u0026#34;); } Process process = processBuilder.start(); int exitCode = process.waitFor(); System.out.println(\u0026#34;Command executed with exit code: \u0026#34; + exitCode); } catch (IOException | InterruptedException e) { e.printStackTrace(); } } } On Windows, setx creates a persistent environment variable. On Unix-like systems, export sets it for the current shell session, but it won\u0026rsquo;t persist across sessions unless you add it to your profile files.\nRemoving Environment Variables Removing environment variables follows similar patterns to creating/updating them:\nFor Child Processes 1 2 ProcessBuilder pb = new ProcessBuilder(\u0026#34;java\u0026#34;, \u0026#34;-version\u0026#34;); pb.environment().remove(\u0026#34;MY_CUSTOM_VAR\u0026#34;); Using JNA 1 int result = JNAEnvSetter.CLibrary.INSTANCE.unsetenv(\u0026#34;MY_ENV_VAR\u0026#34;); Using OS Commands 1 2 3 4 5 6 7 8 9 10 11 // For Windows ProcessBuilder pb = new ProcessBuilder( \u0026#34;cmd\u0026#34;, \u0026#34;/c\u0026#34;, \u0026#34;REG DELETE \\\u0026#34;HKCU\\\\Environment\\\u0026#34; /V MY_ENV_VAR /f\u0026#34; ); // For Unix/Linux/Mac ProcessBuilder pb = new ProcessBuilder( \u0026#34;/bin/sh\u0026#34;, \u0026#34;-c\u0026#34;, \u0026#34;unset MY_ENV_VAR\u0026#34; ); Putting It All Together: A Practical Environment Manager Let me wrap this up with a complete environment variable manager class that you might find useful in your projects:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 import java.io.IOException; import java.util.Map; import java.util.Optional; public class EnvironmentManager { /** * Read a specific environment variable */ public static Optional\u0026lt;String\u0026gt; read(String name) { return Optional.ofNullable(System.getenv(name)); } /** * Read all environment variables */ public static Map\u0026lt;String, String\u0026gt; readAll() { return System.getenv(); } /** * Set an environment variable for child processes */ public static boolean setForChildProcesses(String name, String value) { ProcessBuilder processBuilder = new ProcessBuilder(); processBuilder.environment().put(name, value); return true; } /** * Set a system-wide environment variable */ public static boolean setSystemwide(String name, String value) { try { ProcessBuilder processBuilder; if (isWindows()) { processBuilder = new ProcessBuilder(\u0026#34;cmd\u0026#34;, \u0026#34;/c\u0026#34;, \u0026#34;setx\u0026#34;, name, value); } else { String command = String.format(\u0026#34;export %s=\\\u0026#34;%s\\\u0026#34; \u0026amp;\u0026amp; echo \\\u0026#34;export %s=%s\\\u0026#34; \u0026gt;\u0026gt; ~/.profile\u0026#34;, name, value, name, value); processBuilder = new ProcessBuilder(\u0026#34;/bin/sh\u0026#34;, \u0026#34;-c\u0026#34;, command); } Process process = processBuilder.start(); int exitCode = process.waitFor(); return exitCode == 0; } catch (IOException | InterruptedException e) { e.printStackTrace(); return false; } } /** * Delete an environment variable system-wide */ public static boolean delete(String name) { try { ProcessBuilder processBuilder; if (isWindows()) { processBuilder = new ProcessBuilder( \u0026#34;cmd\u0026#34;, \u0026#34;/c\u0026#34;, \u0026#34;REG DELETE \\\u0026#34;HKCU\\\\Environment\\\u0026#34; /V \u0026#34; + name + \u0026#34; /f\u0026#34; ); } else { String command = String.format(\u0026#34;unset %s \u0026amp;\u0026amp; sed -i \u0026#39;/export %s=/d\u0026#39; ~/.profile\u0026#34;, name, name); processBuilder = new ProcessBuilder(\u0026#34;/bin/sh\u0026#34;, \u0026#34;-c\u0026#34;, command); } Process process = processBuilder.start(); int exitCode = process.waitFor(); return exitCode == 0; } catch (IOException | InterruptedException e) { e.printStackTrace(); return false; } } private static boolean isWindows() { return System.getProperty(\u0026#34;os.name\u0026#34;).toLowerCase().contains(\u0026#34;windows\u0026#34;); } } ","permalink":"https://mohibul.pages.dev/posts/posts/java/100daysofjava/day87/","summary":"Exploring how to read and delete environment variables in Java with practical examples","title":"Day 87: Java's Environment Variable Paradox: Easy to Read, Impossible to Change?"},{"content":"As software engineers we sometimes have to execute some arbitary code that we found or some jar in (Java) case to figure out what its doing. But copy pasting some code in a system directly is a poor security practice thus having a sandbox environment is a powerful tool for ensuring code isolation, enabling secure execution by restricting access to core libraries and system resources. This blog walks through creating a robust sandbox environment in Java using custom class loaders and security policies—tools that are especially valuable for those working on plugins, user-generated scripts, or controlled environments for evaluating code.\nThe Basics of Java Sandboxing Sandboxing in Java involves creating a restricted environment where code can execute safely, isolated from the rest of the system. It allows developers to control which resources are accessible, monitor the classes being loaded, and intercept potentially malicious behaviors. Key components in sandboxing include:\nClass Loaders: Responsible for loading classes into the JVM, custom class loaders enable control over which classes and packages can be accessed. Security Policies: Define the permissions granted to loaded code, restricting access to system properties, file systems, network resources, and other sensitive areas. Modern Java Sandboxing Approaches As of Java 21, there are several approaches to implement secure sandboxing:\nJava Platform Module System (JPMS) - Using strong encapsulation Process Isolation - Running untrusted code in separate JVM processes Custom ClassLoaders with Resource Limits - Our focus for this article System Architecture ┌────────────────────┐\r│ Client Code │\r└─────────┬──────────┘\r│\r┌─────────▼──────────┐\r│ Sandbox Runner │\r├───────────────────-┤\r│ - Resource Limits │\r│ - Class Validation │\r│ - Access Control │\r└─────────┬──────────┘\r│\r┌─────────▼──────────┐\r│ Custom ClassLoader │\r└────────────────────┘ Implementing the Custom Class Loader Our custom class loader, SandboxClassLoader, extends URLClassLoader and overrides the loadClass method to control which classes can be loaded. This ensures that only classes from allowed packages are accessible.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 // filepath: SandboxConfiguration.java public record SandboxConfiguration( Set\u0026lt;String\u0026gt; allowedPackages, long maxMemoryBytes, Duration timeout, Path tempDirectory ) { public static SandboxConfiguration getDefault() { return new SandboxConfiguration( Set.of(\u0026#34;java.lang.\u0026#34;, \u0026#34;java.util.\u0026#34;, \u0026#34;java.math.\u0026#34;), 10_000_000L, // 10MB Duration.ofSeconds(5), Path.of(System.getProperty(\u0026#34;java.io.tmpdir\u0026#34;), \u0026#34;sandbox\u0026#34;) ); } } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 // filepath: ModernSandbox.java public class ModernSandbox implements AutoCloseable { private final SandboxConfiguration config; private final Path tempDir; private final List\u0026lt;String\u0026gt; executionLog; public ModernSandbox(SandboxConfiguration config) { this.config = config; this.tempDir = createTempDirectory(); this.executionLog = new ArrayList\u0026lt;\u0026gt;(); } public ExecutionResult runCode(String sourceCode) { try { // Compile code URL[] urls = compileSource(sourceCode); // Run in controlled environment return ProcessBuilder pb = new ProcessBuilder( \u0026#34;java\u0026#34;, \u0026#34;-Xmx\u0026#34; + config.maxMemoryBytes(), \u0026#34;--add-opens\u0026#34;, \u0026#34;java.base/java.lang=ALL-UNNAMED\u0026#34;, \u0026#34;-cp\u0026#34;, tempDir.toString(), \u0026#34;UntrustedCode\u0026#34; ); // Add timeout control Process process = pb.start(); boolean completed = process.waitFor( config.timeout().toMillis(), TimeUnit.MILLISECONDS ); return new ExecutionResult( completed, process.exitValue(), readOutput(process) ); } catch (Exception e) { return new ExecutionResult(false, -1, e.getMessage()); } } @Override public void close() { // Cleanup temp files FileUtils.deleteDirectory(tempDir.toFile()); } } This policy restricts the code from performing any unauthorized actions, such as accessing the file system or network, by granting only the permissions explicitly added.\nRunning Code in the Sandbox With the custom class loader and security policy in place, we can now execute untrusted code securely.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 import java.lang.reflect.Method; import java.net.URL; public class SandboxRunner { public static void main(String[] args) { // Set the security manager and policy System.setSecurityManager(new SecurityManager()); Policy.setPolicy(new SandboxSecurityPolicy()); String sourceCode = \u0026#34;\u0026#34;\u0026#34; public class UntrustedCode { public static void main(String[] args) { System.out.println(\u0026#34;Hello from untrusted code!\u0026#34;); } } \u0026#34;\u0026#34;\u0026#34;; try { // Compile the source code URL[] classUrls = SandboxUtil.compileSource(sourceCode, \u0026#34;UntrustedCode\u0026#34;); // Create the sandbox class loader SandboxClassLoader sandboxClassLoader = new SandboxClassLoader(classUrls); // Load and run the untrusted class Class\u0026lt;?\u0026gt; untrustedClass = Class.forName(\u0026#34;UntrustedCode\u0026#34;, true, sandboxClassLoader); Method mainMethod = untrustedClass.getMethod(\u0026#34;main\u0026#34;, String[].class); mainMethod.invoke(null, (Object) new String[]{}); } catch (Exception e) { e.printStackTrace(); } } } This SandboxRunner class sets up the security manager and policy, compiles the untrusted code, loads it using the SandboxClassLoader, and invokes its main method.\nTesting the Sandbox Let\u0026rsquo;s test the sandbox with code that should be allowed and code that should be restricted.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 // filepath: ModernSandboxTest.java class ModernSandboxTest { @Test void shouldRestrictFileSystemAccess() { String maliciousCode = \u0026#34;\u0026#34;\u0026#34; public class UntrustedCode { public static void main(String[] args) throws Exception { new java.io.File(\u0026#34;/tmp/evil.txt\u0026#34;).createNewFile(); } } \u0026#34;\u0026#34;\u0026#34;; try (ModernSandbox sandbox = new ModernSandbox(SandboxConfiguration.getDefault())) { ExecutionResult result = sandbox.runCode(maliciousCode); assertFalse(result.successful()); assertTrue(result.output().contains(\u0026#34;SecurityException\u0026#34;)); } } } Real-World Applications Online IDEs: Running user-submitted code safely Plugin Systems: Loading third-party extensions Educational Platforms: Executing student assignments Code Interview Platforms: Running candidate solutions Performance Considerations Process creation: ~100ms overhead Memory usage: Configurable per instance Compilation time: ~50ms for simple classes Conclusion This sandbox implementation provides:\nProcess isolation for maximum security Configurable resource limits Automatic cleanup of temporary files Comprehensive error handling Support for both source files and JARs Perfect for applications requiring secure execution of untrusted code in production environments.\n","permalink":"https://mohibul.pages.dev/posts/posts/java/100daysofjava/day86/","summary":"Implementing a sandbox application that can take JAR or java file and run it securely","title":"Day 86: Building a Secure Java Sandbox with Custom Class Loaders, Process isolation and Java Platform Module System – Executing JAR and .java Files Safely"},{"content":"Idempotency is a reliable way of building robust API, particularly in distributed systems where retries and network failures are common. This blog explores the concept of idempotency and multiple ways to implement it using a Spring Boot file upload service, ensuring no duplicate uploads occur.\nWhat You\u0026rsquo;ll Learn The Concept of Idempotency and why it matters in API design. Different Approaches to Implementing Idempotency in Spring Boot. A Practical File Upload Service showcasing these approaches. Best Practices, Pitfalls, and Optimization Tips for idempotent APIs. Key Takeaways Idempotency Prevents Duplicates: It ensures reliable APIs, handling retries without creating duplicate files. Choose the Right Strategy: Options like Idempotency Keys, Content Hashing, or a combination offer flexibility for various use cases. Content Hashing Simplifies Deduplication: Hashing automates duplicate detection but requires more compute for large files. Combine for Reliability: Pairing Idempotency Keys with Content Hashing is robust and handles client errors effectively. Follow Best Practices: Use strong hashing algorithms, set expiration for keys, and test thoroughly to build resilient APIs. Apply these principles to create efficient, user-friendly, and reliable file upload services.\nThe Problem: Why Idempotency Matters in File Uploads Imagine a scenario where a user uploads a file to your server, but due to network issues, they retry the same request multiple times. Without idempotency, your server might save the same file multiple times, leading to:\nWasted Storage: Duplicate files unnecessarily consume storage. Inconsistent State: Multiple records for the same file can cause confusion. User Frustration: Users might see duplicate entries in their file history. To address this, an idempotent API ensures that repeated uploads of the same file result in only one stored copy.\nApproaches to Implementing Idempotency in Spring Boot Here are four different strategies to implement idempotency for a file upload service in Spring Boot:\n1. Idempotency Key-Based Deduplication How It Works:\nClients include a unique X-Idempotency-Key header with each request. The server stores this key alongside the uploaded file\u0026rsquo;s metadata. If a request with the same key is received again, the server returns the already-stored file\u0026rsquo;s response instead of reprocessing the upload. Implementation Steps:\nStore the idempotencyKey in a database table. Before processing a new upload, check if the key already exists. If the key exists, return the corresponding file metadata. Code Example:\n1 2 3 4 5 6 7 8 9 10 11 public FileRecord uploadFile(String idempotencyKey, MultipartFile file) throws IOException { return fileRecordRepository.findByIdempotencyKey(idempotencyKey) .orElseGet(() -\u0026gt; { String fileHash = DigestUtils.md5Hex(file.getInputStream()); FileRecord fileRecord = new FileRecord(); fileRecord.setFilename(file.getOriginalFilename()); fileRecord.setFileHash(fileHash); fileRecord.setIdempotencyKey(idempotencyKey); return fileRecordRepository.save(fileRecord); }); } Pros:\nSimple and effective. Works well in systems where clients can reliably generate unique keys. Cons:\nRelies on client cooperation to generate unique keys. Requires storage and lookup of keys on the server side. 2. Content Hashing for Deduplication How It Works:\nThe server generates a hash (e.g., MD5 or SHA-256) of the file content. Before saving the file, the server checks if a file with the same hash already exists. If a match is found, the server rejects the upload or returns the existing file\u0026rsquo;s metadata. Implementation Steps:\nGenerate a hash of the file\u0026rsquo;s content. Store the hash in the database alongside the file metadata. Check for hash collisions before processing new uploads. Code Example:\n1 2 3 4 String fileHash = DigestUtils.md5Hex(file.getInputStream()); fileRecordRepository.findByFileHash(fileHash).ifPresent(existing -\u0026gt; { throw new RuntimeException(\u0026#34;Duplicate file upload detected.\u0026#34;); }); Pros:\nAutomatic deduplication based on file content. No reliance on client behavior. Cons:\nComputational overhead for hashing large files. Collisions are rare but theoretically possible (depending on the hashing algorithm). 3. Combination of Idempotency Key and Content Hash How It Works:\nCombines the strengths of both strategies. Clients provide an X-Idempotency-Key, but the server also validates file uniqueness using content hashing. Implementation Steps:\nCheck for the idempotencyKey in the database. If not found, compute the file hash and check for duplicates. Save both the idempotencyKey and file hash in the database. Code Example:\n1 2 3 4 5 6 7 8 9 10 11 12 13 public FileRecord uploadFile(String idempotencyKey, MultipartFile file) throws IOException { return fileRecordRepository.findByIdempotencyKey(idempotencyKey) .orElseGet(() -\u0026gt; { String fileHash = DigestUtils.md5Hex(file.getInputStream()); fileRecordRepository.findByFileHash(fileHash).ifPresent(existing -\u0026gt; { throw new RuntimeException(\u0026#34;Duplicate file upload detected.\u0026#34;); }); FileRecord newRecord = new FileRecord(); newRecord.setIdempotencyKey(idempotencyKey); newRecord.setFileHash(fileHash); return fileRecordRepository.save(newRecord); }); } Pros:\nRobust against client errors (e.g., duplicate idempotency keys). Guarantees no duplicate files. Cons:\nAdds complexity and slightly more overhead. 4. Token-Based Deduplication How It Works:\nAfter a successful upload, the server issues a token (e.g., a UUID or file hash). Clients must include this token in subsequent uploads to reference the original file. If the token is valid, the server skips the upload and returns the original file\u0026rsquo;s response. Implementation Steps:\nGenerate a token after the first successful upload. Store the token alongside the file metadata. Require clients to include the token in subsequent requests. Code Example:\n1 2 3 4 5 6 7 8 9 10 public FileRecord uploadFileWithToken(String token, MultipartFile file) throws IOException { if (fileRecordRepository.existsByToken(token)) { throw new RuntimeException(\u0026#34;Duplicate upload detected.\u0026#34;); } String fileHash = DigestUtils.md5Hex(file.getInputStream()); FileRecord newRecord = new FileRecord(); newRecord.setToken(token); newRecord.setFileHash(fileHash); return fileRecordRepository.save(newRecord); } Pros:\nOffloads responsibility for deduplication to the client. Reduces server-side storage requirements for idempotency keys. Cons:\nPotentially less user-friendly, as clients must manage tokens. Choosing the Right Approach The best strategy depends on your application\u0026rsquo;s requirements:\nCriteria Idempotency Key Content Hash Key + Hash Token Client Cooperation Needed Yes No Partial Yes Storage Efficiency Moderate High Moderate High Computational Overhead Low Medium Medium Low Robustness Against Duplicates Moderate High High High Best Practices for Idempotent File Uploads Use Strong Hashing Algorithms: Prefer SHA-256 over MD5 for better collision resistance. Set Expiry for Idempotency Keys: To avoid unbounded growth in storage, set a TTL (time-to-live) for keys. Test for Edge Cases: Simulate network failures and retries during testing. Provide Clear Error Messages: Inform clients why a request was rejected (e.g., duplicate file detected). Conclusion Idempotency is a critical design principle for modern APIs, ensuring consistency and reliability in distributed systems. By carefully choosing an implementation strategy—such as idempotency keys, content hashing, or a combination—you can build robust file upload services that prevent duplicate uploads and enhance user experience.\nNext Steps: Experiment with the provided code examples. Explore additional use cases for idempotency (e.g., payment processing). Dive deeper into hashing algorithms and database optimization for production-grade systems. By implementing idempotency in your Spring Boot applications, you can create resilient and user-friendly services that handle the challenges of distributed systems gracefully.\n","permalink":"https://mohibul.pages.dev/posts/posts/java/100daysofjava/day85/","summary":"Idempotent APIs in Java: Concepts, Implementation Strategies, and a Spring Boot File Upload Implementation","title":"Day 85: Idempotent APIs in Java: Concepts, Implementation Strategies, and a Spring Boot File Upload Implementation"},{"content":"In the world of software development, time is everything. Every developer knows the frustration of waiting for a full application restart just to see a small change take effect. Java developers, in particular, have long dealt with this issue. But what if you didn’t have to stop and restart every time you updated a class? Enter Hot Class Reload (HCR) in Java—a technique that can keep you in the flow, reloading classes on the fly, much like Hot Module Reload (HMR) does in JavaScript.\nIn this guide, we’ll walk through how to implement HCR and integrate it into your Java development workflow. By the end, you’ll have a powerful new tool to cut down on those long, unproductive restart times.\nUnderstanding Hot Class Reload Simply put, Hot Class Reload (HCR) allows Java applications to reload classes at runtime. This is incredibly useful in development environments where you’re constantly iterating and tweaking code. Think of it as real-time editing: you change some code, the application picks it up right away—no restart required.\nHot Class Reload (HCR) is a technique that enables Java applications to reload classes at runtime. This approach is particularly useful in development environments where frequent code changes occur. By leveraging Java\u0026rsquo;s WatchService, custom class loaders using the javax.tools api and JavaCompiler class we can monitor source files for changes, compile them on-the-fly, and reload the updated classes into the running application.\nHow It Works The core components of our HCR implementation include:\nFile Watcher: Utilizes Java\u0026rsquo;s WatchService to monitor a directory for changes to .java files. Dynamic Compilation: Uses javax.tools.JavaCompiler to compile modified Java source files. Custom Class Loader: Loads the newly compiled classes into the JVM, replacing the old versions. System Flow Diagram Below is a simplified flow diagram illustrating the HCR process:\nSource Changes: Developers modify Java source files in the specified directory. File Watcher: Detects changes and triggers the compilation process. Compile \u0026amp; Reload: Compiles the modified files and reloads the classes using a custom class loader. Custom Class Loader: Loads the newly compiled class into the jvm. Invoke Main Method: Exectues the main method by executing the bytecode of the reloaded class. Implementing Hot Class Reload Here\u0026rsquo;s a breakdown of the Java program implementing HCR:\nSetting Up the Environment The program initializes by setting up directories for source and compiled classes. It ensures these directories exist and are ready for use.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 public HotReload(String sourceDir, String classDir) { this.sourceDir = Paths.get(sourceDir).toAbsolutePath(); this.classDir = Paths.get(classDir).toAbsolutePath(); this.classLoader = new CustomClassLoader(); this.compiler = ToolProvider.getSystemJavaCompiler(); this.fileManager = compiler.getStandardFileManager(null, null, null); // Ensure directories exist try { Files.createDirectories(this.sourceDir); Files.createDirectories(this.classDir); } catch (IOException e) { throw new RuntimeException(\u0026#34;Failed to create directories\u0026#34;, e); } } The constructor begins by converting the provided sourceDir and classDir strings into absolute Path objects using Paths.get().toAbsolutePath(). This ensures that the paths are fully qualified and can be used reliably throughout the application. Next, it initializes the classLoader as an instance of the CustomClassLoader class, which is responsible for dynamically loading compiled classes. The compiler is set to the system Java compiler obtained from ToolProvider.getSystemJavaCompiler(), and the fileManager is initialized using the standard file manager from the compiler. This setup allows the class to compile Java source files programmatically. The constructor then attempts to create the directories specified by sourceDir and classDir if they do not already exist, using Files.createDirectories(). If directory creation fails, it throws a RuntimeException to indicate a critical error that prevents the application from proceeding. This ensures that the necessary directory structure is in place for the class to function correctly.\nWatching for File Changes A dedicated thread runs a file watcher that listens for modifications to .java files in the source directory.The watchForChanges method is responsible for monitoring the source directory for any modifications to .java files and triggering the compileAndReload method when changes are detected. Here is the code for the watchForChanges method:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 private void watchForChanges() { try (WatchService watchService = FileSystems.getDefault() .newWatchService()) { sourceDir.register(watchService, StandardWatchEventKinds.ENTRY_MODIFY); while (true) { WatchKey key = watchService.take(); for (WatchEvent \u0026lt;\u0026lt; ? \u0026gt; event : key.pollEvents()) { if (event.kind() == StandardWatchEventKinds.ENTRY_MODIFY) { Path changed = sourceDir.resolve((Path) event.context()); String fileName = changed.getFileName() .toString(); if (fileName.endsWith(\u0026#34;.java\u0026#34;)) { compileAndReload(fileName); } } } if (!key.reset()) { break; } } } catch (Exception e) { System.err.println(\u0026#34;Error in file watcher: \u0026#34; + e.getMessage()); e.printStackTrace(); } } Compiling and Reloading Classes The compileAndReload method is designed to handle the compilation of a Java source file and subsequently reload the resulting class if necessary. Upon invocation, it accepts a fileName parameter, which should be a Java source file (e.g., \u0026ldquo;MyClass.java\u0026rdquo;). The method begins by extracting the class name by removing the \u0026ldquo;.java\u0026rdquo; extension from the provided file name. It then constructs the paths for both the source file and the compiled class file using sourceDir and classDir, which are assumed to be defined in the broader context of the class.\nTo determine whether recompilation is required, the method compares the modification times of the source file and the corresponding class file. It retrieves the last known modification time from a lastModified map and obtains the current modification time of the source file. The method checks if the class file exists and whether its modification time is newer than or equal to that of the source file. If these conditions indicate that recompilation is unnecessary, the method exits early.\nConversely, if recompilation is deemed necessary, it invokes a separate compile method (details of which are not provided in this snippet) to compile the source file. Upon successful compilation, it calls a reloadClass method (also not shown) to reload the newly compiled class and updates the lastModified map with the current modification time. A success message is printed to confirm the successful operation. In the case of a compilation failure, an error message is displayed. The entire process is encapsulated within a try-catch block to effectively manage any exceptions that may arise during the compilation or file operations.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 private void compileAndReload(String fileName) { try { String className = fileName.replace(\u0026#34;.java\u0026#34;, \u0026#34;\u0026#34;); Path sourceFile = sourceDir.resolve(fileName); Path classFile = classDir.resolve(className + \u0026#34;.class\u0026#34;); // Check if the file has been modified since last compilation long lastModTime = lastModified.getOrDefault(className, 0L); long currentModTime = Files.getLastModifiedTime(sourceFile) .toMillis(); if (Files.exists(classFile) \u0026amp;\u0026amp; Files.getLastModifiedTime(classFile) .toMillis() \u0026gt;= currentModTime \u0026amp;\u0026amp; lastModTime \u0026gt;= currentModTime) { return; } // Compile the Java file boolean compilationSuccessful = compile(sourceFile); if (compilationSuccessful) { // Compilation successful, reload the class reloadClass(className); lastModified.put(className, currentModTime); System.out.println(\u0026#34;Reloaded: \u0026#34; + className); } else { System.err.println(\u0026#34;Compilation failed for: \u0026#34; + className); } } catch (Exception e) { System.err.println(\u0026#34;Error in compile and reload: \u0026#34; + e.getMessage()); e.printStackTrace(); } } The compile method is designed to compile a Java source file utilizing the Java Compiler API, providing a structured approach to dynamic compilation. The method begins by accepting a Path object that represents the source file intended for compilation. To facilitate error handling and reporting, it creates a DiagnosticCollector, which collects diagnostic information such as errors and warnings encountered during the compilation process.\nNext, the method constructs a collection of JavaFileObjects from the source file using a fileManager, likely an instance of StandardJavaFileManager. This setup is crucial for the Java Compiler API to access the source files correctly. The method then specifies compilation options, particularly the output directory for the compiled classes, using the -d option.\nA CompilationTask is created using the Java Compiler API, which is configured with several parameters: no custom writer for compiler output (set to null), the file manager, the diagnostic collector, the compilation options, no annotation processors (also null), and the compilation units (the source files). The compilation task is executed by calling task.call(), which returns a boolean indicating whether the compilation was successful.\nIn the event of a compilation failure, the method iterates through the collected diagnostics, printing detailed error information that includes the line number where the error occurred, the source file URI, and the specific error message. This detailed reporting aids in debugging compilation issues effectively. Finally, the method returns the success status of the compilation, providing a clear indication of the outcome.\nThis method serves as a programmatic solution for compiling Java source files, making it particularly useful in scenarios where dynamic compilation is required. Its robust error reporting capabilities enhance the developer\u0026rsquo;s ability to troubleshoot and resolve compilation errors efficiently.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 private boolean compile(Path sourceFile) { DiagnosticCollector\u0026lt;JavaFileObject\u0026gt; diagnostics = new DiagnosticCollector\u0026lt;\u0026gt;(); Iterable\u0026lt;? extends JavaFileObject\u0026gt; compilationUnits = fileManager.getJavaFileObjectsFromFiles(Collections.singletonList(sourceFile.toFile())); List\u0026lt;String\u0026gt; options = Arrays.asList(\u0026#34;-d\u0026#34;, classDir.toString()); JavaCompiler.CompilationTask task = compiler.getTask(null, fileManager, diagnostics, options, null, compilationUnits); boolean success = task.call(); if (!success) { for (Diagnostic\u0026lt;? extends JavaFileObject\u0026gt; diagnostic : diagnostics.getDiagnostics()) { System.err.format(\u0026#34;Error on line %d in %s%n\u0026#34;, diagnostic.getLineNumber(), diagnostic.getSource() .toUri()); System.err.println(diagnostic.getMessage(null)); } } return success; } Custom Class Loader A custom class loader is used to load the compiled class files into the JVM. The reloading happens after the compilation is successful and then it reloads the class. To reload the class we need to implement a custom class loader that will load the changed class and then execute it.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 private void reloadClass(String className) { try { Class \u0026lt;\u0026lt; ? \u0026gt; loadedClass = classLoader.loadClass(className); Method mainMethod = loadedClass.getMethod(\u0026#34;main\u0026#34;, String[].class); System.out.println(\u0026#34;Invoking main method for: \u0026#34; + className); mainMethod.invoke(null, (Object) new String[0]); } catch (Exception e) { System.err.println(\u0026#34;Error reloading class \u0026#34; + className + \u0026#34;: \u0026#34; + e.getMessage()); e.printStackTrace(); } } private class CustomClassLoader extends ClassLoader { @Override public Class \u0026lt;\u0026lt; ? \u0026gt; loadClass(String name) throws ClassNotFoundException { Path classFile = classDir.resolve(name + \u0026#34;.class\u0026#34;); if (Files.exists(classFile)) { try { byte[] classBytes = Files.readAllBytes(classFile); return defineClass(name, classBytes, 0, classBytes.length); } catch (IOException e) { throw new ClassNotFoundException(\u0026#34;Error loading class \u0026#34; + name, e); } } return super.loadClass(name); } } Now if we want to execute the Hot Reload program we need to do something like this\n1 2 3 4 5 6 7 8 9 10 11 12 13 public static void main(String[] args) { Day84 reloader = new Day84(\u0026#34;C:\\\\Users\\\\mohib\\\\IdeaProjects\\\\date-time\\\\src\\\\main\\\\java\u0026#34;, \u0026#34;C:\\\\Users\\\\mohib\\\\IdeaProjects\\\\date-time\\\\target\\\\classes\u0026#34;); reloader.start(); // Keep the main thread alive and handle user input try (BufferedReader reader = new BufferedReader(new InputStreamReader(System.in))) { System.out.println(\u0026#34;Live reloader is running. Press Enter to exit.\u0026#34;); reader.readLine(); } catch (IOException e) { System.err.println(\u0026#34;Error reading user input: \u0026#34; + e.getMessage()); } System.out.println(\u0026#34;Exiting live reloader.\u0026#34;); } In this main method we can see that we need to provide the path for the src of the code that you want to hot reload and the classes of those java files and after that it then watches for any changes and then compile and reload the class so that we can see the effect immediately.\nTotal code\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 import javax.tools.*; import java.io.*; import java.lang.reflect.Method; import java.nio.file.*; import java.util.*; import java.util.concurrent.ConcurrentHashMap; public class Day84 { private final Path sourceDir; private final Path classDir; private final Map\u0026lt;String, Long\u0026gt; lastModified = new ConcurrentHashMap\u0026lt;\u0026gt;(); private final CustomClassLoader classLoader; private final JavaCompiler compiler; private final StandardJavaFileManager fileManager; public Day84(String sourceDir, String classDir) { this.sourceDir = Paths.get(sourceDir) .toAbsolutePath(); this.classDir = Paths.get(classDir) .toAbsolutePath(); this.classLoader = new CustomClassLoader(); this.compiler = ToolProvider.getSystemJavaCompiler(); this.fileManager = compiler.getStandardFileManager(null, null, null); // Ensure directories exist try { Files.createDirectories(this.sourceDir); Files.createDirectories(this.classDir); } catch (IOException e) { throw new RuntimeException(\u0026#34;Failed to create directories\u0026#34;, e); } } public void start() { Thread watchThread = new Thread(this::watchForChanges, \u0026#34;FileWatcherThread\u0026#34;); watchThread.setDaemon(true); watchThread.start(); System.out.println(\u0026#34;Live reloader started. Watching directory: \u0026#34; + sourceDir); } private void watchForChanges() { try (WatchService watchService = FileSystems.getDefault() .newWatchService()) { sourceDir.register(watchService, StandardWatchEventKinds.ENTRY_MODIFY); while (true) { WatchKey key = watchService.take(); for (WatchEvent\u0026lt;?\u0026gt; event : key.pollEvents()) { if (event.kind() == StandardWatchEventKinds.ENTRY_MODIFY) { Path changed = sourceDir.resolve((Path) event.context()); String fileName = changed.getFileName() .toString(); if (fileName.endsWith(\u0026#34;.java\u0026#34;)) { compileAndReload(fileName); } } } if (!key.reset()) { break; } } } catch (Exception e) { System.err.println(\u0026#34;Error in file watcher: \u0026#34; + e.getMessage()); e.printStackTrace(); } } private void compileAndReload(String fileName) { try { String className = fileName.replace(\u0026#34;.java\u0026#34;, \u0026#34;\u0026#34;); Path sourceFile = sourceDir.resolve(fileName); Path classFile = classDir.resolve(className + \u0026#34;.class\u0026#34;); // Check if the file has been modified since last compilation long lastModTime = lastModified.getOrDefault(className, 0L); long currentModTime = Files.getLastModifiedTime(sourceFile) .toMillis(); if (Files.exists(classFile) \u0026amp;\u0026amp; Files.getLastModifiedTime(classFile) .toMillis() \u0026gt;= currentModTime \u0026amp;\u0026amp; lastModTime \u0026gt;= currentModTime) { return; } // Compile the Java file boolean compilationSuccessful = compile(sourceFile); if (compilationSuccessful) { // Compilation successful, reload the class reloadClass(className); lastModified.put(className, currentModTime); System.out.println(\u0026#34;Reloaded: \u0026#34; + className); } else { System.err.println(\u0026#34;Compilation failed for: \u0026#34; + className); } } catch (Exception e) { System.err.println(\u0026#34;Error in compile and reload: \u0026#34; + e.getMessage()); e.printStackTrace(); } } private boolean compile(Path sourceFile) { DiagnosticCollector\u0026lt;JavaFileObject\u0026gt; diagnostics = new DiagnosticCollector\u0026lt;\u0026gt;(); Iterable\u0026lt;? extends JavaFileObject\u0026gt; compilationUnits = fileManager.getJavaFileObjectsFromFiles(Collections.singletonList(sourceFile.toFile())); List\u0026lt;String\u0026gt; options = Arrays.asList(\u0026#34;-d\u0026#34;, classDir.toString()); JavaCompiler.CompilationTask task = compiler.getTask(null, fileManager, diagnostics, options, null, compilationUnits); boolean success = task.call(); if (!success) { for (Diagnostic\u0026lt;? extends JavaFileObject\u0026gt; diagnostic : diagnostics.getDiagnostics()) { System.err.format(\u0026#34;Error on line %d in %s%n\u0026#34;, diagnostic.getLineNumber(), diagnostic.getSource() .toUri()); System.err.println(diagnostic.getMessage(null)); } } return success; } private void reloadClass(String className) { try { Class\u0026lt;?\u0026gt; loadedClass = classLoader.loadClass(className); Method mainMethod = loadedClass.getMethod(\u0026#34;main\u0026#34;, String[].class); System.out.println(\u0026#34;Invoking main method for: \u0026#34; + className); mainMethod.invoke(null, (Object) new String[0]); } catch (Exception e) { System.err.println(\u0026#34;Error reloading class \u0026#34; + className + \u0026#34;: \u0026#34; + e.getMessage()); e.printStackTrace(); } } private class CustomClassLoader extends ClassLoader { @Override public Class\u0026lt;?\u0026gt; loadClass(String name) throws ClassNotFoundException { Path classFile = classDir.resolve(name + \u0026#34;.class\u0026#34;); if (Files.exists(classFile)) { try { byte[] classBytes = Files.readAllBytes(classFile); return defineClass(name, classBytes, 0, classBytes.length); } catch (IOException e) { throw new ClassNotFoundException(\u0026#34;Error loading class \u0026#34; + name, e); } } return super.loadClass(name); } } public static void main(String[] args) { Day84 reloader = new Day84(\u0026#34;C:\\\\Users\\\\mohib\\\\IdeaProjects\\\\date-time\\\\src\\\\main\\\\java\u0026#34;, \u0026#34;C:\\\\Users\\\\mohib\\\\IdeaProjects\\\\date-time\\\\target\\\\classes\u0026#34;); reloader.start(); // Keep the main thread alive and handle user input try (BufferedReader reader = new BufferedReader(new InputStreamReader(System.in))) { System.out.println(\u0026#34;Live reloader is running. Press Enter to exit.\u0026#34;); reader.readLine(); } catch (IOException e) { System.err.println(\u0026#34;Error reading user input: \u0026#34; + e.getMessage()); } System.out.println(\u0026#34;Exiting live reloader.\u0026#34;); } } Conclusion Hot Class Reload in Java offers a powerful way to enhance development productivity by reducing the need for full application restarts. By integrating file watching, dynamic compilation, and custom class loading, developers can achieve a seamless development experience akin to JavaScript\u0026rsquo;s Hot Module Reload.\nImplementing HCR can significantly speed up the development cycle, allowing for rapid testing and iteration. As you integrate this technique into your workflow, you\u0026rsquo;ll find it an invaluable tool for efficient Java development. There are some libraries like JRebel which provides this feature more on that in the next post.\n","permalink":"https://mohibul.pages.dev/posts/posts/java/100daysofjava/day84/","summary":"Implementing a hot class reload system (HMR for js) using WatchService and javax.tools api","title":"Day 84: Hot Class Reload in Java – A Webpack HMR-like Experience for Java Developers"},{"content":"When working with Java objects, it\u0026rsquo;s crucial to understand how objects can be copied. Objects can be copied either shallowly or deeply, and grasping these concepts is essential when you need to duplicate an object\u0026rsquo;s state without affecting the original object.\nIn this article, we\u0026rsquo;ll explore various methods of creating deep and shallow copies of objects in Java, including some optimized approaches and potential pitfalls to avoid.\nShallow Copy A shallow copy creates a new object but references the same elements as the original object. In other words, the new object contains copies of the values of the original object\u0026rsquo;s fields, but any references to other objects remain the same.\nHere\u0026rsquo;s an example of creating a shallow copy in Java:\n1 2 3 4 5 6 7 8 9 10 11 12 public class Person { private String name; private int age; private List\u0026lt;PhoneNumber\u0026gt; phoneNumbers; public Person(String name, int age, List\u0026lt;PhoneNumber\u0026gt; phoneNumbers) { this.name = name; this.age = age; this.phoneNumbers = phoneNumbers; } // getters \u0026amp; setters } Now let\u0026rsquo;s see how shallow copying works:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 @Test void givenPersonObject_whenShallowCopied_thenCopyHasSameDataButDifferentReference() { List\u0026lt;PhoneNumber\u0026gt; phoneNumbers = new ArrayList\u0026lt;\u0026gt;(); phoneNumbers.add(new PhoneNumber(\u0026#34;123-456-7890\u0026#34;)); Person person1 = new Person(\u0026#34;John Doe\u0026#34;, 30, phoneNumbers); Person person2 = DeepCopyUtil.shallowCopy(person1); assertNotEquals(person1, person2); assertEquals(\u0026#34;John Doe\u0026#34;, person1.getName()); assertEquals(\u0026#34;John Doe\u0026#34;, person2.getName()); person2.setName(\u0026#34;Jane Doe\u0026#34;); assertEquals(\u0026#34;John Doe\u0026#34;, person1.getName()); assertNotEquals(\u0026#34;John Doe\u0026#34;, person2.getName()); person2.getPhoneNumbers().add(new PhoneNumber(\u0026#34;456-789-0123\u0026#34;)); assertEquals(2, person1.getPhoneNumbers().size()); } In this example, when we assign person1 to person2, we\u0026rsquo;re creating a shallow copy. As a result, both person1 and person2 refer to the same Person object in memory. Modifying the phoneNumbers property of person2 also affects the phoneNumbers property of person1.\nShallow Copy Methods There are several ways to create shallow copies in Java:\nObject.clone() method: Requires implementing the Cloneable interface.\n1 2 3 4 5 6 7 8 public class Person implements Cloneable { // ... fields and other methods ... @Override public Object clone() throws CloneNotSupportedException { return super.clone(); } } Copy Constructor: Can be used for shallow copying if it only copies references.\n1 2 3 4 public Person(Person other) { this.name = other.name; // Shallow copy of reference this.age = other.age; } Does Using Clone() make a Deep Copy? The clone() method typically creates a shallow copy, not a deep copy. This means that while it creates a new object, it doesn\u0026rsquo;t create new instances of the mutable objects contained within the original object.\nHere\u0026rsquo;s an example using the Cloneable interface:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 public class PersonCloneable implements Cloneable { private String name; private int age; private List\u0026lt;PhoneNumber\u0026gt; phoneNumbers; public PersonCloneable(String name, int age, List\u0026lt;PhoneNumber\u0026gt; phoneNumbers) { this.name = name; this.age = age; this.phoneNumbers = phoneNumbers; } @Override public PersonCloneable clone() throws CloneNotSupportedException { return (PersonCloneable) super.clone(); } // Getters \u0026amp; setters } Now let\u0026rsquo;s look at a test to understand it better:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 @Test void givenPersonCloneableObject_whenCloned_thenShallowCopyIsCreated() throws CloneNotSupportedException { List\u0026lt;PhoneNumber\u0026gt; phoneNumbers = new ArrayList\u0026lt;\u0026gt;(); phoneNumbers.add(new PhoneNumber(\u0026#34;123-456-7890\u0026#34;)); PersonCloneable original = new PersonCloneable(\u0026#34;John Doe\u0026#34;, 30, phoneNumbers); PersonCloneable clone = original.clone(); clone.setName(\u0026#34;Jane Doe\u0026#34;); clone.setAge(25); clone.getPhoneNumbers().add(new PhoneNumber(\u0026#34;987-654-3210\u0026#34;)); assertEquals(\u0026#34;John Doe\u0026#34;, original.getName()); assertEquals(30, original.getAge()); assertEquals(2, original.getPhoneNumbers().size()); assertEquals(2, clone.getPhoneNumbers().size()); } In this example, we use the clone() method to create a shallow copy of the PersonCloneable object. This creates a new PersonCloneable object with its own memory location, independent of the original person1 object.\nHowever, modifying the phoneNumbers property of the clone affects the phoneNumbers of the original. That\u0026rsquo;s one of the potential issues of using clone(), as it makes shallow copies, not deep copies.\nPotential Issues with Clone() While the clone() method can be used to create copies of objects, it has some significant issues:\nNeeds Cloneable interface: To use the clone() method, a class must explicitly implement the Cloneable interface. By default, the clone() method is protected in the Object class, and if a class doesn\u0026rsquo;t implement it, trying to use clone() will result in a CloneNotSupportedException.\nDefault Shallow Copy: The default clone() method in the Object class creates a shallow copy. This means that if the object contains references to other mutable objects, the cloned object will still share those references with the original, rather than duplicating the referenced objects themselves.\nSecurity Concerns: Overriding the clone() method allows for custom cloning behavior, but it also introduces potential security risks. Here\u0026rsquo;s a more detailed explanation:\nImagine you\u0026rsquo;re at a copy machine in an office. You\u0026rsquo;re supposed to make an exact copy of a document, but someone has tampered with the machine. Instead of making a true copy, it might add or remove pages, or even worse, send a copy to someone else! That\u0026rsquo;s similar to what can happen with the clone() method if it\u0026rsquo;s not implemented correctly.\nHere\u0026rsquo;s why:\nUnexpected Behavior: When you override clone(), you\u0026rsquo;re in charge of deciding exactly how the copying happens. If you\u0026rsquo;re not careful, you might accidentally expose private data or create inconsistent objects.\nThe \u0026ldquo;Fake Copy\u0026rdquo; Problem: A malicious subclass could override clone() to return a reference to itself instead of creating a new object. This could trick other parts of the code into thinking they have a fresh copy when they don\u0026rsquo;t.\nBypassing Security Checks: Normally, when you create an object, its constructor runs and can perform important security checks. But clone() bypasses the constructor, potentially creating objects in an insecure state.\nInconsistent State: If clone() is interrupted (like by an exception), it might leave the new object half-created. This could lead to bugs or vulnerabilities if the code isn\u0026rsquo;t prepared to handle partially-cloned objects.\nHere\u0026rsquo;s a simple example to illustrate:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 class SecretKeeper implements Cloneable { private String secretPassword = \u0026#34;1234\u0026#34;; @Override public SecretKeeper clone() { try { SecretKeeper cloned = (SecretKeeper) super.clone(); // Oops! We forgot to create a new String for secretPassword // Now the clone shares the same String reference as the original return cloned; } catch (CloneNotSupportedException e) { return null; } } public void changePassword(String newPassword) { this.secretPassword = newPassword; } } // In some other part of the code... SecretKeeper original = new SecretKeeper(); SecretKeeper cloned = original.clone(); // The programmer thinks this only changes the clone\u0026#39;s password cloned.changePassword(\u0026#34;5678\u0026#34;); // But surprise! It changed the original\u0026#39;s password too // because both objects are sharing the same String reference In this example, a programmer using the SecretKeeper class might think they\u0026rsquo;re safely changing only the cloned object\u0026rsquo;s password, but they\u0026rsquo;re actually changing the original object\u0026rsquo;s password too. This could lead to serious security issues in a real application.\nTo avoid these pitfalls, many Java developers prefer to use copy constructors or factory methods for object copying. These approaches give you more control and make it easier to ensure that you\u0026rsquo;re creating secure, consistent copies of objects.\nDeep Copy Methods To perform a deep copy in Java, we have multiple options such as using a custom copy method or a copy constructor. Furthermore, we can leverage serialization and third-party libraries for this purpose. Below, we\u0026rsquo;ll discuss deep copy methods including custom copy methods and serialization.\nCustom Copy Method One way to create a deep copy is to manually create new instances of all mutable objects. Here\u0026rsquo;s an example:\n1 2 3 public Person deepCopy() { return new Person(new String(this.name), this.age); } Serialization Java serialization is one of the most efficient ways of creating deep copies of an object. This approach works well for serializable objects (those that implement the Serializable interface). Using third-party libraries like Jackson and Gson, you can also do this.\nFor example, let\u0026rsquo;s modify the Person class to make it serializable:\n1 2 3 4 5 6 7 8 9 10 11 public class PersonSerializable implements Serializable { private String name; private int age; private List\u0026lt;PhoneNumber\u0026gt; phoneNumbers; public PersonSerializable(String name, int age, List\u0026lt;PhoneNumber\u0026gt; phoneNumbers) { this.name = name; this.age = age; this.phoneNumbers = phoneNumbers; } } Now our custom deepCopy() method will use serialization-deserialization to copy an object:\n1 2 3 4 5 6 7 8 9 10 11 public static \u0026lt;T extends Serializable\u0026gt; T deepCopy(T object) { try { ByteArrayOutputStream baos = new ByteArrayOutputStream(); ObjectOutputStream oos = new ObjectOutputStream(baos); oos.writeObject(object); ObjectInputStream ois = new ObjectInputStream(new ByteArrayInputStream(baos.toByteArray())); return (T) ois.readObject(); } catch (IOException | ClassNotFoundException e) { throw new IllegalArgumentException(e); } } Here we are making deep copies of an object by serializing it into a byte array and then deserializing it from that byte array. This approach works well for creating a deep copy but requires that all objects in the graph are Serializable.\nNow let\u0026rsquo;s see its use:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 @Test void givenPersonSerializableObject_whenDeepCopied_thenCopyIsIndependentOfOriginal() { List\u0026lt;PhoneNumber\u0026gt; phoneNumbers = new ArrayList\u0026lt;\u0026gt;(); phoneNumbers.add(new PhoneNumber(\u0026#34;123-456-7890\u0026#34;)); PersonSerializable original = new PersonSerializable(\u0026#34;John Doe\u0026#34;, 30, phoneNumbers); PersonSerializable copy = DeepCopyUtil.deepCopy(original); assertNotSame(original, copy); assertEquals(original.getName(), copy.getName()); assertEquals(original.getAge(), copy.getAge()); copy.setName(\u0026#34;Jane Doe\u0026#34;); copy.setAge(25); copy.getPhoneNumbers().add(new PhoneNumber(\u0026#34;987-654-3210\u0026#34;)); assertEquals(\u0026#34;John Doe\u0026#34;, original.getName()); assertEquals(30, original.getAge()); assertEquals(1, original.getPhoneNumbers().size()); assertEquals(2, copy.getPhoneNumbers().size()); } This method serializes the object to a byte array and then deserializes it to create a new instance. From the test, we can see that it can effectively create a deep copy of the PersonSerializable object.\nOther Deep Copy Methods JSON Serialization/Deserialization: Use libraries like Jackson or Gson.\n1 2 ObjectMapper mapper = new ObjectMapper(); Person copy = mapper.readValue(mapper.writeValueAsString(original), Person.class); Cloning Libraries: Use third-party libraries like Apache Commons Lang.\n1 Person copy = SerializationUtils.clone(original); Using Reflection:\nAnother powerful technique for creating deep copies is using reflection. This method allows you to create a deep copy of an object without the need for implementing Cloneable or writing custom constructors. It works by recursively copying the fields of an object, even if the fields are private or inaccessible.\nHere\u0026rsquo;s an example of how to implement deep copying using reflection:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 import java.lang.reflect.Field; import java.util.HashMap; import java.util.Map; public class DeepCopyUtil { // A map to track objects and their deep copies to handle cyclic dependencies private static Map\u0026lt;Object, Object\u0026gt; visited = new HashMap\u0026lt;\u0026gt;(); public static Object deepCopy(Object object) throws IllegalAccessException, InstantiationException { // Check if the object is already copied (to handle cyclic dependencies) if (visited.containsKey(object)) { return visited.get(object); // Return the already-copied object } // If the object is null, no need to copy if (object == null) { return null; } Class\u0026lt;?\u0026gt; clazz = object.getClass(); Object copy = clazz.newInstance(); // Create a new instance of the object visited.put(object, copy); // Register the object and its copy in the map // Copy each field of the object for (Field field : clazz.getDeclaredFields()) { field.setAccessible(true); Object value = field.get(object); // Check if a deep copy is required if (value != null \u0026amp;\u0026amp; isDeepCopyRequired(value)) { // Recursively copy the field, ensuring cyclic dependencies are handled field.set(copy, deepCopy(value)); } else { // Directly copy the field if it\u0026#39;s a primitive, string, or number field.set(copy, value); } } return copy; } private static boolean isDeepCopyRequired(Object value) { return !(value.getClass().isPrimitive() || value instanceof String || value instanceof Number); } public static void clearVisitedMap() { visited.clear(); // Clear the map after deep copying to avoid memory leaks } } This method works by:\nCreating a new instance of the object\u0026rsquo;s class. Iterating through all declared fields of the object. For each field, if it\u0026rsquo;s not a primitive, String, or Number (which are immutable), it recursively applies the deep copy method. Otherwise, it simply copies the value. Here the method might produce cyclic dependencies which can be resolved by checking if we already created the object. Thats why we have a visited map in the example to resolve that. Let\u0026rsquo;s see how we can use this method:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 public class Person { String name; Person friend; // Cyclic reference possible public Person(String name) { this.name = name; } } @Test public void testDeepCopyWithCyclicDependency() throws Exception { Person john = new Person(\u0026#34;John\u0026#34;); Person jane = new Person(\u0026#34;Jane\u0026#34;); // Create cyclic dependency john.friend = jane; jane.friend = john; // Deep copy the object Person johnCopy = (Person) DeepCopyUtil.deepCopy(john); // Ensure deep copy is successful and cyclic references are handled Assertions.assertNotSame(john, johnCopy); Assertions.assertNotSame(jane, johnCopy.friend); Assertions.assertSame(johnCopy, johnCopy.friend.friend); // Clear the visited map to avoid memory leak DeepCopyUtil.clearVisitedMap(); } This reflection-based deep copying technique has several advantages and disadvantages:\nPros: It works without modifying your class or implementing interfaces. It can copy private and protected fields. It\u0026rsquo;s a generic solution that can work with any object.\nCons: This approach can be slower due to the use of reflection. It might require handling some special cases, such as cyclic dependencies. It may not work correctly with final fields or certain complex objects.\nWhen using this method, be aware of its limitations and test thoroughly with your specific object structures. Also, consider the performance implications, especially when dealing with large object graphs or in performance-critical sections of your code.\nOptimized Approaches Copy-on-Write: For collections, Java provides copy-on-write implementations like CopyOnWriteArrayList.\n1 2 3 4 5 CopyOnWriteArrayList\u0026lt;Person\u0026gt; list = new CopyOnWriteArrayList\u0026lt;\u0026gt;(); // Adding elements doesn\u0026#39;t create a copy list.add(new Person(\u0026#34;John\u0026#34;, 30)); // But modifying operations do list.set(0, new Person(\u0026#34;Jane\u0026#34;, 28)); Immutable Objects: If objects are immutable, shallow copying is effectively the same as deep copying.\n1 2 3 4 5 6 7 8 9 10 11 public final class ImmutablePerson { private final String name; private final int age; public ImmutablePerson(String name, int age) { this.name = name; this.age = age; } // Only getters, no setters } Lazy Copying: Implement a copy-on-write mechanism for individual fields.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 public class LazyPerson { private String name; private int age; private LazyPerson original; public LazyPerson(String name, int age) { this.name = name; this.age = age; } public LazyPerson(LazyPerson other) { this.original = other; } public String getName() { if (original != null) { return original.getName(); } return name; } public void setName(String name) { if (original != null) { this.name = original.name; this.age = original.age; original = null; } this.name = name; } // Similar for age } Conclusion In this article, we\u0026rsquo;ve covered what shallow and deep copy means in Java. Shallow copies create new objects but maintain references to the same nested objects, while deep copies create entirely new objects with independent states, including nested objects.\nWe\u0026rsquo;ve explored various methods for creating both shallow and deep copies, including the clone() method, custom copy methods, serialization, and some optimized approaches. We\u0026rsquo;ve also discussed the potential security issues with using clone() and why many developers prefer alternative methods.\nWhen choosing a copying method, consider the specific requirements of your application, including performance needs, object complexity, and maintenance considerations. Always profile your application to determine the most efficient method for your specific scenario.\nRemember, the \u0026ldquo;best\u0026rdquo; approach often depends on the specific requirements of your application. Shallow copying might be sufficient for simple objects or when performance is critical, while deep copying is necessary when you need to create truly independent copies of complex object graphs.\n","permalink":"https://mohibul.pages.dev/posts/posts/java/100daysofjava/day83/","summary":"Exploring shallow and deep copying of object in java","title":"Day 83: Java Object Cloning: Exploring Shallow and Deep Copy Techniques"},{"content":"Reflection in Java is a powerful concept that serves as the backbone for numerous Java and J2EE frameworks [1]. This feature allows an executing Java program to examine or introspect upon itself, and manipulate its internal properties at runtime [2]. While it\u0026rsquo;s of limited use in everyday programming, reflection plays a crucial role in various frameworks that lack direct access to user-defined classes, interfaces, and their methods.\nWhat is this introspect Imagine giving a program a magic mirror that lets it see and understand itself. That\u0026rsquo;s what introspection does in Java. It\u0026rsquo;s like the code is able to figure out what it is and what it can do, all while it\u0026rsquo;s running.\nThis self-examination superpower is super handy. It helps debuggers peek inside code, powers those handy autocompletions in your IDE, and even helps convert data between different formats. It\u0026rsquo;s all thanks to some clever Java tools that let programs ask themselves questions like \u0026ldquo;What am I?\u0026rdquo; or \u0026ldquo;What can I do?\u0026rdquo; While it\u0026rsquo;s a cool trick, it needs to be used carefully. Overdo it, and your program might slow down or accidentally reveal things meant to be secret. But when used right, it\u0026rsquo;s like giving your code X-ray vision – a powerful tool that opens up a world of possibilities for flexible, dynamic programming.\nUnderstanding Reflection API The Reflection API provides developers with the ability to analyze and modify class fields and methods during runtime [1]. It offers several methods to examine Class fields, invoke methods, and create new instances of classes by invoking constructors [1]. To use reflection, developers typically follow three steps:\nObtain a java.lang.Class object for the class to manipulate. Use the Class object to gather information about the class. Utilize reflection classes like Method from java.lang.reflect to perform operations. One practical application of reflection is in JavaBeans, where software components can be manipulated visually via builder tools [2]. These tools use reflection to obtain the properties of Java components as they are dynamically loaded. Reflections another practical application\nReflection for Runtime Type Introspection Reflection enables runtime type introspection, allowing programs to examine and interact with objects whose types may not be known at compile time. This capability is particularly useful in scenarios where dynamic behavior is required.\nFor instance, reflection can be used to:\nSimulate the instanceof operator using Class.isInstance method [2]. Discover methods defined within a class [2]. Examine constructors and data fields of a class [2]. Invoke methods dynamically, even when the method name is specified during execution [2]. Modify values of data fields in objects at runtime [2]. These capabilities make reflection a powerful tool for creating flexible and adaptable software systems. However, it\u0026rsquo;s important to use reflection judiciously, as it can impact performance and potentially break encapsulation principles.\nSecurity Considerations with Reflection While reflection offers significant flexibility, it also introduces security considerations that developers must address: Potential for Code Injection: Unsafe use of reflection mechanisms can create unexpected control flow paths through an application, potentially bypassing security checks [3]. This vulnerability may allow attackers to create limited forms of code injection.\nAccess Control Risks: Reflection can be used to access private fields and methods, potentially breaking encapsulation and leaking sensitive data [4]. This capability can lead to various malicious exploits, such as modifying supposedly immutable strings or revealing sensitive information from objects [4].\nClassloader Hierarchy Concerns: In shared hosting environments like Google App Engine, reflection could potentially be used to enumerate classes and methods from different users\u0026rsquo; WAR files if not properly restricted [4].\nTo mitigate these risks, developers should consider the following precautions:\nInstall and configure a SecurityManager to enforce access controls Use the setAccessible() method judiciously, as it\u0026rsquo;s subject to security checks governed by the setAccessChecks reflection permission [4]. Be cautious when using reflection to instantiate objects or invoke methods, especially with user-supplied input [3]. By understanding both the power and the potential risks of reflection, developers can leverage this feature effectively while maintaining the security and integrity of their applications.\nExploring Dependency Injection Patterns Dependency Injection (DI) is a design pattern in Java that aims to decouple classes from their dependencies, making code more flexible, testable, and maintainable [5]. It allows developers to move the dependency resolution from compile-time to runtime, resulting in loosely coupled and extendable applications [6]. There are several patterns for implementing dependency injection in Java, each with its own advantages and use cases.\nConstructor Injection Constructor injection involves passing dependencies as arguments to a class\u0026rsquo;s constructor. This method ensures that all required dependencies are available upon object creation, making it ideal for mandatory dependencies [5]. Constructor injection supports immutability and state safety, as the object is either instantiated with a full state or not instantiated at all [7].\n1 2 3 4 5 6 7 8 9 10 11 public class Car { private final Engine engine; private final Steering steering; @Autowired //Spring like @inject //JAX-RS public Car(Engine engine, Steering steering) { thisותs.engine = engine; this.steering = steering; } } This pattern is particularly useful when dealing with components that are essential for an object\u0026rsquo;s functionality, such as a car needing an engine and steering [8].\nSetter Injection Setter injection utilizes setter methods to inject dependencies after object creation. This approach offers more flexibility but can make the class mutable [5]. It\u0026rsquo;s preferred for optional dependencies that are not mandatorily required but can assist in some ways [8].\n1 2 3 4 5 6 7 8 public class User { private String phoneNumber; @Autowired public void setPhoneNumber(String phoneNumber) { this.phoneNumber = phoneNumber; } } Setter injection is suitable for scenarios where dependencies may change during the object\u0026rsquo;s lifecycle or when dealing with optional components [7].\nMethod Injection Method injection involves injecting dependencies directly into methods where they are needed. This pattern is less common but can be useful in specific scenarios where dependencies are required only for certain operations.\n1 2 3 4 5 6 public class MessageSender { @Autowired public void sendMessage(MessageService messageService, String message) { messageService.send(message); } } Field Injection Field injection directly injects dependencies into class fields, typically using annotations. While it results in less boilerplate code and can improve readability, it doesn\u0026rsquo;t support immutability and can make testing more challenging [7].\n1 2 3 4 public class EmailService { @Autowired private EmailClient emailClient; } When choosing a dependency injection pattern, developers should consider factors such as immutability, state safety, and the nature of the dependencies (mandatory vs. optional). Constructor injection is generally recommended for its support of immutability and clear dependency declaration, while setter injection offers more flexibility for optional dependencies [7].\nTo implement these patterns effectively, developers can use frameworks like Spring, which provide powerful dependency injection containers. These containers manage the creation and injection of dependencies, inverting the control flow and allowing for more modular and maintainable code [5].\nDesigning a Lightweight DI Container Designing a lightweight Dependency Injection (DI) container involves creating a framework that manages the creation and lifecycle of objects in an application. This approach allows developers to implement Inversion of Control (IoC), a design principle that separates the responsibility of creating and managing dependencies from the client classes [9]. A well-designed DI container enhances modularity, reusability, and testability of code by promoting loose coupling between components [10].\nContainer Initialization The first step in designing a lightweight DI container is to create an initialization mechanism. This process typically involves: Defining an AbstractBinder or similar class to specify how dependencies should be created and managed [11]. Registering the binder with the application, often through a configuration file or programmatically [11]. Implementing a class that extends a framework-specific class (e.g., ResourceConfig in JAX-RS) to register the binder and specify the packages to scan for injectable components [11].\nFor example, in a JAX-RS application, the initialization might look like this:\n1 2 3 4 5 6 public class MyApplication extends ResourceConfig { public MyApplication() { register(new MyApplicationBinder()); packages(true, \u0026#34;com.mypackage.rest\u0026#34;); } } This initialization sets up the container to manage dependencies within the specified package and its subpackages.\nDependency Scanning and Registration Once the container is initialized, it needs to scan for and register dependencies. This process involves:\nScanning all clients under the root package and subpackages [9]. Creating instances of client classes [9]. Identifying services used in client classes, including member variables, constructor parameters, and method parameters [9]. Recursively scanning for nested dependencies within services [9]. Creating a map of all client classes and their dependencies [9]. During this phase, the container should also handle special cases, such as:\nValidating if there are multiple implementations of an interface or no implementation at all [9]. Managing qualifiers for services or implementing autowiring by type when multiple implementations exist [9]. Resolving and Injecting Dependencies The final step in the DI process is resolving and injecting dependencies. This involves:\nCreating instances of services identified during the scanning phase [9]. Injecting (initializing) each service with its required dependencies [9]. Providing methods to retrieve beans or services, such as getBean(Class classz) or getService(Class classz) [9]. The injection process follows a specific order as defined by JSR330 [12]: Constructor injection 2.Field injection 3.Method injection It\u0026rsquo;s important to note that the order in which methods or fields annotated with @Inject are called is not defined by JSR330, so developers should not assume a specific order of injection [12]. To implement this lightweight DI container, developers can use annotations like @Inject to mark injection points [13]. For example:\n1 2 3 4 5 6 7 8 9 public class Payroll { private EmployeeDatabase employeeDatabase; @Inject public Payroll(EmployeeDatabase employeeDatabase) { this.employeeDatabase = employeeDatabase; } // ... rest of the class implementation } This design allows for flexible configuration and easy testing, as dependencies can be easily mocked or stubbed [10]. It also promotes the use of immutable objects and makes dependencies explicit, which enhances code maintainability [10]. By implementing these components, developers can create a lightweight DI container that provides the benefits of dependency injection without the complexity of larger frameworks. This approach is particularly useful for standalone Java applications or when retrofitting legacy systems as part of major refactoring efforts [14].\nNow lets implement a DI container\nAt first we will see how the container is made up and then will create the services and load them in the container and use them to see.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 public class DIContainer { private final Map\u0026lt;Class\u0026lt;?\u0026gt;, Object\u0026gt; singletons = new ConcurrentHashMap\u0026lt;\u0026gt;(); private final Map\u0026lt;Class\u0026lt;?\u0026gt;, Class\u0026lt;?\u0026gt;\u0026gt; implementations = new ConcurrentHashMap\u0026lt;\u0026gt;(); public \u0026lt;T\u0026gt; void bindImplementation(Class\u0026lt;T\u0026gt; interfaceClass, Class\u0026lt;? extends T\u0026gt; implementationClass) { implementations.put(interfaceClass, implementationClass); } public \u0026lt;T\u0026gt; T getInstance(Class\u0026lt;T\u0026gt; clazz) { try { return getOrCreateInstance(clazz); } catch (Exception e) { throw new RuntimeException(\u0026#34;Error creating instance of \u0026#34; + clazz.getName(), e); } } @SuppressWarnings(\u0026#34;unchecked\u0026#34;) private \u0026lt;T\u0026gt; T getOrCreateInstance(Class\u0026lt;T\u0026gt; clazz) throws Exception { Class\u0026lt;?\u0026gt; targetClass = implementations.getOrDefault(clazz, clazz); if (targetClass.isAnnotationPresent(Singleton.class)) { return (T) singletons.computeIfAbsent(targetClass, this::createInstance); } return createInstance(targetClass); } @SuppressWarnings(\u0026#34;unchecked\u0026#34;) private \u0026lt;T\u0026gt; T createInstance(Class\u0026lt;?\u0026gt; clazz) { try { Constructor\u0026lt;?\u0026gt; constructor = findInjectableConstructor(clazz); T instance = (T) (constructor != null ? createInstanceWithConstructor(constructor) : clazz.getDeclaredConstructor().newInstance()); injectFields(clazz, instance); injectMethods(clazz, instance); return instance; } catch (Exception e) { throw new RuntimeException(\u0026#34;Error creating instance of \u0026#34; + clazz.getName(), e); } } private Constructor\u0026lt;?\u0026gt; findInjectableConstructor(Class\u0026lt;?\u0026gt; clazz) { return Arrays.stream(clazz.getDeclaredConstructors()) .filter(c -\u0026gt; c.isAnnotationPresent(Inject.class)) .findFirst() .orElse(null); } private Object createInstanceWithConstructor(Constructor\u0026lt;?\u0026gt; constructor) throws Exception { Object[] params = Arrays.stream(constructor.getParameterTypes()) .map(this::getInstance) .toArray(); return constructor.newInstance(params); } private void injectFields(Class\u0026lt;?\u0026gt; clazz, Object instance) throws Exception { for (Field field : clazz.getDeclaredFields()) { if (field.isAnnotationPresent(Inject.class)) { Object fieldInstance = getInstance(field.getType()); field.setAccessible(true); field.set(instance, fieldInstance); } } } private void injectMethods(Class\u0026lt;?\u0026gt; clazz, Object instance) throws Exception { for (Method method : clazz.getDeclaredMethods()) { if (method.isAnnotationPresent(Inject.class)) { Object[] params = Arrays.stream(method.getParameterTypes()) .map(this::getInstance) .toArray(); method.setAccessible(true); method.invoke(instance, params); } } } } now let us break this class down. The DIContainer class is the core of our DI framework. It uses two main data structures:\nsingletons: A thread-safe map to store singleton instances implementations: A map to store interface-to-implementation bindings I use annotations and reflection to make the core features of the DI.\nThe bindImplementation method allows users to specify which implementation should be used for a given interface: This is useful when you want to inject an interface but use a specific implementation.\n1 2 3 public \u0026lt;T\u0026gt; void bindImplementation(Class\u0026lt;T\u0026gt; interfaceClass, Class\u0026lt;? extends T\u0026gt; implementationClass) { implementations.put(interfaceClass, implementationClass); } The public getInstance method is the entry point for retrieving instances: It delegates to getOrCreateInstance, wrapping any exceptions in a RuntimeException.\n1 2 3 4 5 6 7 public \u0026lt;T\u0026gt; T getInstance(Class\u0026lt;T\u0026gt; clazz) { try { return getOrCreateInstance(clazz); } catch (Exception e) { throw new RuntimeException(\u0026#34;Error creating instance of \u0026#34; + clazz.getName(), e); } } The getOrCreateInstance method is where the magic happens: It first checks if there\u0026rsquo;s a registered implementation for the requested class. If the target class is annotated with @Singleton, it uses computeIfAbsent to ensure thread-safe lazy initialization. Otherwise, it creates a new instance each time.\n1 2 3 4 5 6 7 8 9 private \u0026lt;T\u0026gt; T getOrCreateInstance(Class\u0026lt;T\u0026gt; clazz) throws Exception { Class\u0026lt;?\u0026gt; targetClass = implementations.getOrDefault(clazz, clazz); if (targetClass.isAnnotationPresent(Singleton.class)) { return (T) singletons.computeIfAbsent(targetClass, this::createInstance); } return createInstance(targetClass); } The createInstance method handles the actual object creation: It performs these steps:\nFind an @Inject-annotated constructor Create the instance using the constructor or default constructor Inject fields Inject methods 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 private \u0026lt;T\u0026gt; T createInstance(Class\u0026lt;?\u0026gt; clazz) { try { Constructor\u0026lt;?\u0026gt; constructor = findInjectableConstructor(clazz); T instance = (T) (constructor != null ? createInstanceWithConstructor(constructor) : clazz.getDeclaredConstructor().newInstance()); injectFields(clazz, instance); injectMethods(clazz, instance); return instance; } catch (Exception e) { throw new RuntimeException(\u0026#34;Error creating instance of \u0026#34; + clazz.getName(), e); } } Constructor Injection findInjectableConstructor looks for a constructor annotated with @Inject: If found, createInstanceWithConstructor handles the dependency injection: This method searches for a constructor annotated with @Inject. Here\u0026rsquo;s what\u0026rsquo;s happening:\nWe get all declared constructors of the class using clazz.getDeclaredConstructors() a reflection method that gets declared constructors. We create a stream from this array of constructors We filter the stream to keep only constructors annotated with @Inject We take the first matching constructor (findFirst()) If no constructor is found, we return null (orElse(null)) This approach allows us to specify which constructor should be used for dependency injection. If no @Inject-annotated constructor is found, we\u0026rsquo;ll fall back to using the default constructor.\n1 2 3 4 5 6 private Constructor\u0026lt;?\u0026gt; findInjectableConstructor(Class\u0026lt;?\u0026gt; clazz) { return Arrays.stream(clazz.getDeclaredConstructors()) .filter(c -\u0026gt; c.isAnnotationPresent(Inject.class)) .findFirst() .orElse(null); } createInstanceWithConstructor() This method creates an instance using the provided constructor. Here\u0026rsquo;s the process:\nWe get the parameter types of the constructor We create a stream of these types For each type, we call getInstance() to get or create an instance of that type We collect these instances into an array Finally, we invoke the constructor with these parameters.This recursive approach ensures that all dependencies (and their dependencies) are properly instantiated. 1 2 3 4 5 6 private Object createInstanceWithConstructor(Constructor\u0026lt;?\u0026gt; constructor) throws Exception { Object[] params = Arrays.stream(constructor.getParameterTypes()) .map(this::getInstance) .toArray(); return constructor.newInstance(params); } Field Injection The injectFields method handles field injection:\nWe iterate over all declared fields of the class for each field annotated with @Inject\nWe get an instance of the field\u0026rsquo;s type using getInstance() We make the field accessible (in case it\u0026rsquo;s private) We set the field\u0026rsquo;s value on the instance to our newly created fieldInstance This allows for dependency injection on fields, which can be useful for optional dependencies or when constructor injection isn\u0026rsquo;t feasible.\n1 2 3 4 5 6 7 8 9 private void injectFields(Class\u0026lt;?\u0026gt; clazz, Object instance) throws Exception { for (Field field : clazz.getDeclaredFields()) { if (field.isAnnotationPresent(Inject.class)) { Object fieldInstance = getInstance(field.getType()); field.setAccessible(true); field.set(instance, fieldInstance); } } } Method Injection This method performs method injection: We iterate over all declared methods of the class\nFor each method annotated with @Inject:\nWe get the parameter types of the method We create instances for each parameter type using getInstance() We make the method accessible We invoke the method on the instance with the created parameters Method injection can be useful for optional dependencies or for performing post-construction initialization.\n1 2 3 4 5 6 7 8 9 10 11 private void injectMethods(Class\u0026lt;?\u0026gt; clazz, Object instance) throws Exception { for (Method method : clazz.getDeclaredMethods()) { if (method.isAnnotationPresent(Inject.class)) { Object[] params = Arrays.stream(method.getParameterTypes()) .map(this::getInstance) .toArray(); method.setAccessible(true); method.invoke(instance, params); } } } This DIContainer implementation provides a flexible and powerful dependency injection framework. It supports constructor, field, and method injection, as well as singleton management and interface-to-implementation binding. The use of reflection allows for a clean and non-invasive way to manage dependencies in your application.\nNow let us see the Main class and how to use it.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 interface MessageService { void sendMessage(String message); } @Singleton class EmailService implements MessageService { public void sendMessage(String message) { System.out.println(\u0026#34;Sending email: \u0026#34; + message); } } class UserService { private final MessageService messageService; @Inject public UserService(MessageService messageService) { this.messageService = messageService; } public void notifyUser(String message) { messageService.sendMessage(message); } } public class Main { public static void main(String[] args) { DIContainer container = new DIContainer(); container.bindImplementation(MessageService.class, EmailService.class); UserService userService = container.getInstance(UserService.class); userService.notifyUser(\u0026#34;Hello from, DI!\u0026#34;); } } This example demonstrates a basic Dependency Injection (DI) setup using a custom container:\nComponent structure:\nMessageService: An interface defining the contract for messaging. EmailService: A concrete @Singleton implementation of MessageService. UserService: A service with a dependency on MessageService. DI configuration:\nUserService uses constructor injection, marked with @Inject. The DI container is configured to bind MessageService to EmailService. Runtime behavior:\nAn DIContainer instance is created and configured. When getInstance(UserService.class) is called, the container: a. Recognizes the MessageService dependency. b. Instantiates EmailService (or reuses the singleton instance). c. Injects the EmailService into UserService. d. Returns the constructed UserService. Key DI principles demonstrated:\nInversion of Control (IoC): UserService doesn\u0026rsquo;t create its dependencies. Loose coupling: UserService depends on the MessageService interface, not the concrete EmailService. Single Responsibility Principle: The DI container manages object creation and wiring. Benefits:\nImproved testability: Dependencies can be easily mocked. Flexibility: Changing implementations (e.g., to an SMSService) requires only container reconfiguration. Separation of concerns: Business logic is separated from object construction and wiring. This setup showcases core DI concepts, illustrating how it promotes loosely coupled, modular, and easily maintainable code architecture.\nTo wrap up, reflection in Java proves to be a game-changer for building flexible and adaptable applications. Its ability to enable runtime introspection and modification has a profound impact on the development of frameworks and tools, particularly in the realm of dependency injection. The creation of a lightweight DI container showcases the practical application of reflection, demonstrating how it can be harnessed to implement sophisticated design patterns and improve code modularity. The exploration of various dependency injection patterns and the design considerations for a DI container highlight the power of reflection to create more maintainable and testable code. By leveraging annotations and reflection, developers can craft systems that are not only more flexible but also easier to extend and modify over time. This approach to software design opens up new possibilities for creating robust and scalable applications, ultimately leading to more efficient development processes and higher-quality software products.\nReferences [1] - https://www.digitalocean.com/community/tutorials/java-reflection-example-tutorial [2] - https://www.oracle.com/technical-resources/articles/java/javareflection.html [3] - https://owasp.org/www-community/vulnerabilities/Unsafe_use_of_Reflection [4] - https://stackoverflow.com/questions/3002904/what-is-the-security-risk-of-object-reflection [5] - https://www.geeksforgeeks.org/dependency-injection-di-design-pattern/ [6] - https://www.digitalocean.com/community/tutorials/java-dependency-injection-design-pattern-example-tutorial [7] - https://www.javacodegeeks.com/2019/02/field-setter-constructor-injection.html [8] - https://www.geeksforgeeks.org/spring-setter-injection-vs-constructor-injection/ [9] - https://dev.to/jjbrt/how-to-create-your-own-dependency-injection-framework-in-java-4eaj [10] - https://www.geeksforgeeks.org/what-is-the-best-way-to-inject-dependency-in-java/ [11] - https://stackoverflow.com/questions/16216759/dependency-injection-with-jersey-2-0 [12] - https://www.vogella.com/tutorials/DependencyInjection/article.html [13] - https://www.objc.io/issue-11/dependency-injection-in-java.html [14] - https://stackoverflow.com/questions/1333438/is-there-a-simple-framework-allowing-for-dependency-injection-in-a-stand-alone-p\n","permalink":"https://mohibul.pages.dev/posts/posts/java/100daysofjava/day82/","summary":"making a minimal spring like java dependency injection container using java reflection and annotation api","title":"Day 82: Reflection in Java: Building a Simple DI Container in 100 Lines of Code"},{"content":"Custom Spliterator in Java: Enhancing Stream Operations In the world of Java development, mastering the Streams API is crucial for efficient data processing. However, there are times when the built-in functionalities fall short of our specific needs. This is where custom Spliterators come into play. In this blog post, we\u0026rsquo;ll explore how to create a custom Spliterator that implements shifting, sliding, zipping, and folding operations on streams, significantly enhancing your data processing capabilities.\nUnderstanding Spliterators Before we dive into the implementation, let\u0026rsquo;s briefly recap what a Spliterator is. Short for \u0026ldquo;splitable iterator,\u0026rdquo; a Spliterator is an object that traverses and partitions elements of a source. It\u0026rsquo;s designed to work seamlessly with the Stream API and supports parallel processing.\nWhy Custom Spliterators Matter Custom Spliterators allow us to:\nExtend stream functionality beyond built-in operations Optimize performance for specific data processing patterns Combine multiple operations efficiently Create reusable, composable stream components Implementing a Custom Spliterator Our CustomSpliterator class will implement four key operations:\nShifting: Cyclically move elements in the stream Sliding: Create windows of elements that slide through the stream Zipping: Combine elements from two streams Folding: Apply a cumulative operation to stream elements Let\u0026rsquo;s break down each operation:\n1. Shift Operation 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 public static \u0026lt;T\u0026gt; Stream\u0026lt;T\u0026gt; shift(Stream\u0026lt;T\u0026gt; stream, int offset) { Iterator\u0026lt;T\u0026gt; iterator = stream.iterator(); Queue\u0026lt;T\u0026gt; queue = new LinkedList\u0026lt;\u0026gt;(); for (int i = 0; i \u0026lt; offset \u0026amp;\u0026amp; iterator.hasNext(); i++) { queue.add(iterator.next()); } return StreamSupport.stream(new Spliterators.AbstractSpliterator\u0026lt;T\u0026gt;(Long.MAX_VALUE, Spliterator.ORDERED) { @Override public boolean tryAdvance(Consumer\u0026lt;? super T\u0026gt; action) { if (iterator.hasNext()) { queue.add(iterator.next()); action.accept(queue.poll()); return true; } return !queue.isEmpty() \u0026amp;\u0026amp; queue.poll() != null; } }, false); } This method shifts the elements in the stream by a specified offset. It uses a queue to hold the shifted elements and advances the stream accordingly.\n2. Slide Operation 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 public static \u0026lt;T\u0026gt; Stream\u0026lt;List\u0026lt;T\u0026gt;\u0026gt; slide(Stream\u0026lt;T\u0026gt; stream, int windowSize) { Iterator\u0026lt;T\u0026gt; iterator = stream.iterator(); Deque\u0026lt;T\u0026gt; deque = new LinkedList\u0026lt;\u0026gt;(); for (int i = 0; i \u0026lt; windowSize - 1 \u0026amp;\u0026amp; iterator.hasNext(); i++) { deque.add(iterator.next()); } return StreamSupport.stream(new Spliterators.AbstractSpliterator\u0026lt;List\u0026lt;T\u0026gt;\u0026gt;(Long.MAX_VALUE, Spliterator.ORDERED) { @Override public boolean tryAdvance(Consumer\u0026lt;? super List\u0026lt;T\u0026gt;\u0026gt; action) { if (iterator.hasNext()) { deque.add(iterator.next()); if (deque.size() \u0026gt; windowSize) { deque.pollFirst(); } action.accept(new ArrayList\u0026lt;\u0026gt;(deque)); return true; } return false; } }, false); } The slide operation creates overlapping windows of elements. It maintains a deque of the current window and slides it through the stream.\n3. Zip Operation 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 public static \u0026lt;A, B\u0026gt; Stream\u0026lt;AbstractMap.SimpleEntry\u0026lt;A, B\u0026gt;\u0026gt; zip(Stream\u0026lt;A\u0026gt; streamA, Stream\u0026lt;B\u0026gt; streamB) { Spliterator\u0026lt;A\u0026gt; spliteratorA = streamA.spliterator(); Spliterator\u0026lt;B\u0026gt; spliteratorB = streamB.spliterator(); Iterator\u0026lt;A\u0026gt; iteratorA = Spliterators.iterator(spliteratorA); Iterator\u0026lt;B\u0026gt; iteratorB = Spliterators.iterator(spliteratorB); return StreamSupport.stream(new Spliterators.AbstractSpliterator\u0026lt;AbstractMap.SimpleEntry\u0026lt;A, B\u0026gt;\u0026gt;( Math.min(spliteratorA.estimateSize(), spliteratorB.estimateSize()), Spliterator.ORDERED) { @Override public boolean tryAdvance(Consumer\u0026lt;? super AbstractMap.SimpleEntry\u0026lt;A, B\u0026gt;\u0026gt; action) { if (iteratorA.hasNext() \u0026amp;\u0026amp; iteratorB.hasNext()) { A a = iteratorA.next(); B b = iteratorB.next(); action.accept(new AbstractMap.SimpleEntry\u0026lt;\u0026gt;(a, b)); return true; } return false; } }, false); } The zip operation combines elements from two streams into pairs. It advances both streams simultaneously, creating pairs until one of the streams is exhausted.\n4. Fold Operation 1 2 3 4 5 6 7 8 public static \u0026lt;T, R\u0026gt; R fold(Stream\u0026lt;T\u0026gt; stream, R identity, BiFunction\u0026lt;R, T, R\u0026gt; accumulator) { Iterator\u0026lt;T\u0026gt; iterator = stream.iterator(); R result = identity; while (iterator.hasNext()) { result = accumulator.apply(result, iterator.next()); } return result; } The fold operation, also known as reduce, applies a cumulative operation to the stream elements. It\u0026rsquo;s a powerful tool for aggregating data.\nFull Code: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 import java.util.*; import java.util.function.*; import java.util.stream.*; public class CustomSpliterator { public static \u0026lt;T\u0026gt; Stream\u0026lt;T\u0026gt; shift(Stream\u0026lt;T\u0026gt; stream, int offset) { Iterator\u0026lt;T\u0026gt; iterator = stream.iterator(); Queue\u0026lt;T\u0026gt; queue = new LinkedList\u0026lt;\u0026gt;(); for (int i = 0; i \u0026lt; offset \u0026amp;\u0026amp; iterator.hasNext(); i++) { queue.add(iterator.next()); } return StreamSupport.stream(new Spliterators.AbstractSpliterator\u0026lt;T\u0026gt;(Long.MAX_VALUE, Spliterator.ORDERED) { @Override public boolean tryAdvance(Consumer\u0026lt;? super T\u0026gt; action) { if (iterator.hasNext()) { queue.add(iterator.next()); action.accept(queue.poll()); return true; } return !queue.isEmpty() \u0026amp;\u0026amp; queue.poll() != null; } }, false); } public static \u0026lt;T\u0026gt; Stream\u0026lt;List\u0026lt;T\u0026gt;\u0026gt; slide(Stream\u0026lt;T\u0026gt; stream, int windowSize) { Iterator\u0026lt;T\u0026gt; iterator = stream.iterator(); Deque\u0026lt;T\u0026gt; deque = new LinkedList\u0026lt;\u0026gt;(); for (int i = 0; i \u0026lt; windowSize - 1 \u0026amp;\u0026amp; iterator.hasNext(); i++) { deque.add(iterator.next()); } return StreamSupport.stream(new Spliterators.AbstractSpliterator\u0026lt;List\u0026lt;T\u0026gt;\u0026gt;(Long.MAX_VALUE, Spliterator.ORDERED) { @Override public boolean tryAdvance(Consumer\u0026lt;? super List\u0026lt;T\u0026gt;\u0026gt; action) { if (iterator.hasNext()) { deque.add(iterator.next()); if (deque.size() \u0026gt; windowSize) { deque.pollFirst(); } action.accept(new ArrayList\u0026lt;\u0026gt;(deque)); return true; } return false; } }, false); } public static \u0026lt;A, B\u0026gt; Stream\u0026lt;AbstractMap.SimpleEntry\u0026lt;A, B\u0026gt;\u0026gt; zip(Stream\u0026lt;A\u0026gt; streamA, Stream\u0026lt;B\u0026gt; streamB) { Spliterator\u0026lt;A\u0026gt; spliteratorA = streamA.spliterator(); Spliterator\u0026lt;B\u0026gt; spliteratorB = streamB.spliterator(); Iterator\u0026lt;A\u0026gt; iteratorA = Spliterators.iterator(spliteratorA); Iterator\u0026lt;B\u0026gt; iteratorB = Spliterators.iterator(spliteratorB); return StreamSupport.stream(new Spliterators.AbstractSpliterator\u0026lt;AbstractMap.SimpleEntry\u0026lt;A, B\u0026gt;\u0026gt;( Math.min(spliteratorA.estimateSize(), spliteratorB.estimateSize()), Spliterator.ORDERED) { @Override public boolean tryAdvance(Consumer\u0026lt;? super AbstractMap.SimpleEntry\u0026lt;A, B\u0026gt;\u0026gt; action) { if (iteratorA.hasNext() \u0026amp;\u0026amp; iteratorB.hasNext()) { A a = iteratorA.next(); B b = iteratorB.next(); action.accept(new AbstractMap.SimpleEntry\u0026lt;\u0026gt;(a, b)); return true; } return false; } }, false); } public static \u0026lt;T, R\u0026gt; R fold(Stream\u0026lt;T\u0026gt; stream, R identity, BiFunction\u0026lt;R, T, R\u0026gt; accumulator) { Iterator\u0026lt;T\u0026gt; iterator = stream.iterator(); R result = identity; while (iterator.hasNext()) { result = accumulator.apply(result, iterator.next()); } return result; } } here is the runner class\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 public class Day81 { public static void main(String[] args) { System.out.println(\u0026#34;Shift example\u0026#34;); List\u0026lt;Integer\u0026gt; list = Arrays.asList(1, 2, 3, 4, 5); Stream\u0026lt;Integer\u0026gt; shiftedStream = CustomSpliterator.shift(list.stream(), 2); shiftedStream.forEach(System.out::println); System.out.println(\u0026#34;Slide example\u0026#34;); Stream\u0026lt;List\u0026lt;Integer\u0026gt;\u0026gt; slidingStream = CustomSpliterator.slide(list.stream(), 3); slidingStream.forEach(System.out::println); System.out.println(\u0026#34;Zip example\u0026#34;); List\u0026lt;String\u0026gt; list2 = Arrays.asList(\u0026#34;a\u0026#34;, \u0026#34;b\u0026#34;, \u0026#34;c\u0026#34;); Stream\u0026lt;AbstractMap.SimpleEntry\u0026lt;Integer, String\u0026gt;\u0026gt; zippedStream = CustomSpliterator.zip(list.stream(), list2.stream()); zippedStream.forEach(pair -\u0026gt; System.out::println(pair.getKey() + \u0026#34;, \u0026#34; + pair.getValue())); System.out.println(\u0026#34;Fold example\u0026#34;); Integer sum = CustomSpliterator.fold(list.stream(), 0, Integer::sum); System.out.println(\u0026#34;Sum: \u0026#34; + sum); } } Example: Calculating Moving Average Here\u0026rsquo;s a practical example of using our custom Spliterator to calculate a moving average:\n1 2 3 4 5 6 7 List\u0026lt;Double\u0026gt; stockPrices = Arrays.asList(100.0, 102.0, 98.0, 103.0, 99.0, 101.0); int windowSize = 3; Stream\u0026lt;Double\u0026gt; movingAverages = CustomSpliterator.slide(stockPrices.stream(), windowSize) .map(window -\u0026gt; window.stream().mapToDouble(Double::doubleValue).average().orElse(0.0)); movingAverages.forEach(System.out::println); This code efficiently calculates the 3-day moving average of stock prices using our custom sliding window operation.\nCreating a custom Spliterator is an advanced technique that can significantly enhance your Java data processing capabilities. By combining shifting, sliding windows, zipping, and folding into a single, efficient component, we\u0026rsquo;ve created a powerful tool for complex data manipulation tasks.\nAs you implement these techniques in your projects, remember to consider the trade-offs. While custom Spliterators offer great flexibility and power, they can also introduce complexity. Always benchmark your implementations to ensure they provide the expected performance benefits.\nHappy coding, and may your streams flow efficiently!\n","permalink":"https://mohibul.pages.dev/posts/posts/java/100daysofjava/day81/","summary":"Implementing Sliding, shifting, Zipping and Folding a Stream by creating it from a custom spliterator","title":"Day 81: Creating custom stream operations ex: Slide, Shift, Zip and Fold using Spliterator"},{"content":"As a software engineer working with distributed systems, ensuring efficient data distribution across nodes is crucial. Consistent hashing offers a robust solution for this challenge. This blog post will delve into a Java implementation of consistent hashing with unit tests, along with the key concepts and design decisions involved.\nUnderstanding Consistent Hashing Traditional hashing techniques often result in significant rehashing of keys when nodes are added or removed. Consistent hashing mitigates this by ensuring that only a small subset of keys are remapped. This property makes it highly suitable for scalable and fault-tolerant distributed systems.\nKey Concepts Fig: Consistent Hashing\nHash Function A Hash Function is a mathematical algorithm that takes an input (or \u0026lsquo;message\u0026rsquo;) and returns a fixed-size string of bytes. The output is typically a hash code or hash value. The primary purpose of a hash function is to map data of arbitrary size to data of a fixed size. In the context of consistent hashing, the hash function is used to assign both data keys and node identifiers to positions on the hash ring.\nThe effectiveness of a hash function in consistent hashing depends on its ability to uniformly distribute the keys across the hash ring, minimizing the chances of collisions where two different inputs produce the same hash value. A good hash function ensures that the keys are spread evenly, which leads to efficient load balancing among nodes.\nHash Ring The Hash Ring is a conceptual circular space that represents the range of possible hash values produced by the hash function. Imagine a circle where the hash values wrap around, creating a continuous loop. Both nodes (servers, caches, etc.) and data keys are hashed and placed on this ring.\nWhen placing a data key on the ring, the system looks for the nearest node in the clockwise direction. This node is responsible for storing that key. The circular nature of the hash ring ensures that every key has a node assignment, and it allows for a smooth transition when nodes are added or removed. By using a circular space, consistent hashing efficiently handles node dynamics with minimal data reshuffling.\nVirtual Nodes Virtual Nodes (or \u0026ldquo;vnodes\u0026rdquo;) are logical subdivisions of physical nodes in the hash ring. Instead of each physical node being represented by a single point on the ring, it can be represented by multiple virtual nodes. Each virtual node is assigned a unique position on the hash ring.\nVirtual nodes enhance the distribution of keys across the physical nodes, especially when there are a small number of physical nodes. They help achieve a more balanced load by mitigating the effects of uneven key distribution and ensuring that no single physical node becomes a bottleneck. When a physical node is added or removed, only the virtual nodes associated with that physical node need to be redistributed, leading to better fault tolerance and load balancing.\nReplication Replication in the context of consistent hashing refers to the practice of placing multiple copies of each node on the hash ring to ensure better distribution and fault tolerance. This is controlled by the numberOfReplicas parameter in our implementation.\nBy replicating nodes, we can distribute the keys more evenly across the hash ring. Each physical node is represented by several points (virtual nodes), and each point can independently take responsibility for a portion of the data. This approach also enhances fault tolerance: if a physical node fails, its virtual nodes can be easily reassigned to other physical nodes, ensuring that the data remains accessible. Replication minimizes the impact of node failures and ensures that the system can continue to function smoothly even when changes occur.\nProblems Solved by Consistent Hashing Scalability: Allows the system to scale easily by adding or removing nodes with minimal impact on data distribution.\nLoad Balancing: Distributes data evenly across nodes, preventing any single node from becoming a bottleneck.\nFault Tolerance: Reduces the risk of data loss by redistributing data automatically when nodes fail.\nImplementing Consistent Hashing in Java We begin by defining the ConsistentHashing class. This class includes methods to add and remove nodes and to find the corresponding node for a given key. We use the SHA-256 algorithm for hashing to ensure a good distribution of hash values.\nAt first here I am considering\nWeighted Nodes: Each node\u0026rsquo;s capacity is taken into account by adjusting the number of virtual nodes based on the node\u0026rsquo;s capacity. Nodes with higher capacities have more virtual nodes, ensuring a more balanced load distribution. Dynamic Node Management: Methods to add and remove nodes dynamically, taking into account the node capacities and redistributing data as needed. Replication and Quorum: Ensures data availability and consistency by writing to a quorum of nodes and reading from a quorum of nodes, even in the presence of node failures or removals. This enhanced implementation ensures that nodes with different capacities are effectively utilized, leading to a more balanced and efficient distributed system. To implement this I am using a TreeMap. which uses the Red-Black tree inside it. Lets understand it a little bit.\nKey Characteristics of TreeMap Ordered: TreeMap maintains its elements in ascending order, according to the natural ordering of its keys or by a comparator provided at map creation time.\nNavigableMap Features: It provides several methods to navigate the map, such as firstKey, lastKey, lowerKey, higherKey, subMap, headMap, and tailMap.\nInternal Working Red-Black Tree: Internally, TreeMap uses a Red-Black tree, which is a self-balancing binary search tree. This ensures that the tree remains balanced, and the operations can be performed in logarithmic time.\nNode Structure: Each node in the tree contains: Key Value References to the left child, right child, and parent nodes A color attribute (red or black) to maintain balance according to Red-Black tree properties Key Characteristics of Red-Black tree A Red-Black Tree is a type of self-balancing binary search tree. Each node in the tree contains an extra bit for storing its color, which can be either red or black. The color attribute helps maintain the balance of the tree during insertions and deletions. Here’s a detailed explanation:\nKey Properties of Red-Black Trees Binary Search Tree Property: Like all binary search trees (BST), for each node, the value of the left child is less than its value, and the value of the right child is greater.\nRed-Black Properties:\nProperty 1: Each node is either red or black. Property 2: The root node is always black. Property 3: All leaves (NIL nodes) are black. These are the null children of a leaf node, considered black for simplicity in maintaining tree properties. Property 4: If a red node has children, then the children are always black (no two red nodes can be adjacent). Property 5: Every path from a given node to any of its descendant leaves has the same number of black nodes (black-height). Now let us see the use case of TreeMap to implement a Dummy consistent hashing\n1 2 3 4 5 6 7 8 public class ConsistentHashingWithWeightedNodes { private final SortedMap\u0026lt;Integer, String\u0026gt; ring = new TreeMap\u0026lt;\u0026gt;(); private final Map\u0026lt;String, Integer\u0026gt; nodeCapacities = new HashMap\u0026lt;\u0026gt;(); private final int replicationFactor; private final int writeQuorum; private final int readQuorum; private final Map\u0026lt;String, Map\u0026lt;String, String\u0026gt;\u0026gt; dataStore = new HashMap\u0026lt;\u0026gt;(); private final MessageDigest md5; This defines the ConsistentHashingWithWeightedNodes class with the following fields:\nring: A SortedMap to represent the consistent hashing ring. nodeCapacities: A Map to store each node\u0026rsquo;s capacity. replicationFactor: Number of replicas for each data item. writeQuorum: Minimum number of nodes that must acknowledge a write for it to be considered successful. readQuorum: Minimum number of nodes that must be contacted to successfully read data. dataStore: A nested Map to store the actual data on each node. md5: An instance of MessageDigest for computing MD5 hashes. 1 2 3 4 5 public ConsistentHashingWithWeightedNodes(Map\u0026lt;String, Integer\u0026gt; nodes, int replicationFactor, int writeQuorum, int readQuorum) throws NoSuchAlgorithmException { this.md5 = MessageDigest.getInstance(\u0026#34;MD5\u0026#34;); this.replicationFactor = replicationFactor; this.writeQuorum = writeQuorum; this.readQuorum = readQuorum; This is the constructor of the ConsistentHashingWithWeightedNodes class. It initializes the md5 instance for hashing and sets the replication factor, write quorum, and read quorum.\n1 2 3 4 for (Map.Entry\u0026lt;String, Integer\u0026gt; entry : nodes.entrySet()) { addNode(entry.getKey(), entry.getValue()); } } The constructor adds all nodes from the input nodes map, which contains node names and their respective capacities, to the consistent hashing ring using the addNode method.\n1 2 3 4 5 private int hash(String key) { md5.update(key.getBytes()); byte[] digest = md5.digest(); return ByteBuffer.wrap(digest).getInt(); } This method computes the MD5 hash of a given key and converts it to an integer. The hash value is used to place nodes and data on the consistent hashing ring. Here the ByteBuffer is used for getting the output of digest array to an interger.\nLet’s say the digest array contains the following bytes (in hexadecimal):\n1 digest = {0x12, 0x34, 0x56, 0x78, ...}; When you call ByteBuffer.wrap(digest).getInt() the following steps occur:\nWrap the Byte Array: ByteBuffer.wrap(digest) creates a ByteBuffer that points to the digest array.\nRead an Integer: getInt() reads the first four bytes: 0x12, 0x34, 0x56, 0x78. Since ByteBuffer uses big-endian order, these bytes are interpreted as the integer:0x12345678\nIn decimal, this value is 305419896.\nSo, the line ByteBuffer.wrap(digest).getInt() effectively converts the first four bytes of the digest array into an integer in a straightforward and readable manner.\nUsage in Context In the context of the hash function, this conversion is used to transform the MD5 hash (which is a 16-byte array) into a single integer value. This integer value is then used to place nodes and keys on the consistent hashing ring.\n1 2 3 public void addNode(String node, int capacity) { nodeCapacities.put(node, capacity); int virtualNodes = capacity; // Use capacity directly as the number of virtual nodes for simplicity The addNode method adds a node to the consistent hashing ring. The number of virtual nodes for this node is determined by its capacity.\n1 2 3 4 5 for (int i = 0; i \u0026lt; virtualNodes; i++) { int hash = hash(node + \u0026#34;#\u0026#34; + i); ring.put(hash, node); } } For each virtual node, the method computes its hash and adds it to the ring.\n1 2 public void removeNode(String node) { int virtualNodes = nodeCapacities.get(node); The removeNode method removes a node and its virtual nodes from the ring. It first retrieves the number of virtual nodes based on the node\u0026rsquo;s capacity.\n1 2 3 4 5 6 7 for (int i = 0; i \u0026lt; virtualNodes; i++) { int hash = hash(node + \u0026#34;#\u0026#34; + i); ring.remove(hash); } nodeCapacities.remove(node); } It then removes each virtual node from the ring and finally removes the node from the nodeCapacities map.\n1 2 3 4 private List\u0026lt;String\u0026gt; getNodesForKey(String key) { List\u0026lt;String\u0026gt; nodes = new ArrayList\u0026lt;\u0026gt;(); int hash = hash(key); SortedMap\u0026lt;Integer, String\u0026gt; tailMap = ring.tailMap(hash); The getNodesForKey method determines which nodes should store a given key. It first computes the hash of the key and finds the relevant segment in the ring.\n1 2 3 4 5 6 7 8 for (Map.Entry\u0026lt;Integer, String\u0026gt; entry : tailMap.entrySet()) { if (!nodes.contains(entry.getValue())) { nodes.add(entry.getValue()); if (nodes.size() == replicationFactor) { return nodes; } } } It iterates through the tail map (the part of the ring starting from the key\u0026rsquo;s hash) and adds nodes to the list until the replication factor is met.\n1 2 3 4 5 6 7 8 9 10 11 for (Map.Entry\u0026lt;Integer, String\u0026gt; entry : ring.entrySet()) { if (!nodes.contains(entry.getValue())) { nodes.add(entry.getValue()); if (nodes.size() == replicationFactor) { break; } } } return nodes; } If the tail map does not contain enough nodes, it continues from the beginning of the ring until it has enough nodes to meet the replication factor.\n1 2 3 4 5 6 7 8 9 10 11 12 13 public boolean writeData(String key, String value) { List\u0026lt;String\u0026gt; nodes = getNodesForKey(key); int successCount = 0; for (String node : nodes) { if (writeToNode(node, key, value)) { successCount++; if (successCount \u0026gt;= writeQuorum) { return true; } } } return false; } The writeData method writes data to the nodes responsible for the key. It retrieves the nodes using getNodesForKey and writes to each node until the write quorum is met.\n1 2 3 4 5 6 7 private boolean writeToNode(String node, String key, String value) { if (!dataStore.containsKey(node)) { dataStore.put(node, new HashMap\u0026lt;\u0026gt;()); } dataStore.get(node).put(key, value); return true; } The writeToNode method actually writes the key-value pair to the node\u0026rsquo;s data store. If the node does not already have a data store, it creates one.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 public String readData(String key) { List\u0026lt;String\u0026gt; nodes = getNodesForKey(key); Map\u0026lt;String, Integer\u0026gt; valueCounts = new HashMap\u0026lt;\u0026gt;(); for (String node : nodes) { String value = readFromNode(node, key); if (value != null) { valueCounts.put(value, valueCounts.getOrDefault(value, 0) + 1); if (valueCounts.get(value) \u0026gt;= readQuorum) { return value; } } } return \u0026#34;No Data Found\u0026#34;; } The readData method reads data for a given key from the responsible nodes. It counts the occurrences of each value read and returns the value if it meets the read quorum.\n1 2 3 4 5 6 private String readFromNode(String node, String key) { if (dataStore.containsKey(node)) { return dataStore.get(node).get(key); } return null; } The readFromNode method retrieves the value for a key from a specific node\u0026rsquo;s data store.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 public static void main(String[] args) throws NoSuchAlgorithmException { Map\u0026lt;String, Integer\u0026gt; nodes = new HashMap\u0026lt;\u0026gt;(); nodes.put(\u0026#34;Node1\u0026#34;, 1); // Capacity 1 nodes.put(\u0026#34;Node2\u0026#34;, 3); // Capacity 3 nodes.put(\u0026#34;Node3\u0026#34;, 2); // Capacity 2 int replicationFactor = 3; int writeQuorum = 2; int readQuorum = 2; ConsistentHashingWithWeightedNodes ch = new ConsistentHashingWithWeightedNodes(nodes, replicationFactor, writeQuorum, readQuorum); System.out.println(\u0026#34;Writing data...\u0026#34;); System.out.println(\u0026#34;Write key1: \u0026#34; + ch.writeData(\u0026#34;key1\u0026#34;, \u0026#34;value1\u0026#34;)); System.out.println(\u0026#34;Write key2: \u0026#34; + ch.writeData(\u0026#34;key2\u0026#34;, \u0026#34;value2\u0026#34;)); System.out.println(\u0026#34;Write key3: \u0026#34; + ch.writeData(\u0026#34;key3\u0026#34;, \u0026#34;value3\u0026#34;)); System.out.println(\u0026#34;Reading data...\u0026#34;); System.out.println(\u0026#34;Read key1: \u0026#34; + ch.readData(\u0026#34;key1\u0026#34;)); System.out.println(\u0026#34;Read key2: \u0026#34; + ch.readData(\u0026#34;key2\u0026#34;)); System.out.println(\u0026#34;Read key3: \u0026#34; + ch.readData(\u0026#34;key3\u0026#34;)); ch.addNode(\u0026#34;Node4\u0026#34;, 5); // Adding a new node with higher capacity System.out.println(\u0026#34;Write key4 after adding Node4: \u0026#34; + ch.writeData(\u0026#34;key4\u0026#34;, \u0026#34;value4\u0026#34;)); ch.removeNode(\u0026#34;Node2\u0026#34;); System.out.println(\u0026#34;Read key2 after removing Node2: \u0026#34; + ch.readData(\u0026#34;key2\u0026#34;)); } } The main method demonstrates the usage of the ConsistentHashingWithWeightedNodes class. It:\nCreates a map of nodes with different capacities. Initializes the consistent hashing ring with these nodes. Writes some data and prints the results. Reads the data Here I discussed about Replication Factor and Quorum. Will discuss them in details in some later post. For now posting some resource for further learning consistent hashing, Red-Black tree, Replication and Quorum.\nConsistent Hashing : https://medium.com/codex/understanding-and-implementing-consistent-hash-algorithm-e53a35afa428 https://rmarcus.info/blog/2018/09/14/consistent-hashing-overflow.html Red-Black Tree : https://zarif98sjs.github.io/blog/blog/redblacktree/ https://stackoverflow.com/questions/31779786/how-tree-map-uses-red-black-tree-algorithm Replication In the context of Consistent hashing : https://dev.to/arslan_ah/how-to-use-consistent-hashing-in-a-system-design-interview-33ge https://www.geeksforgeeks.org/what-is-replication-in-consistent-hashing/ Quorum : https://systemdesign.one/consistency-patterns/?ref=highscalability.com https://code.likeagirl.io/distributed-computing-quorum-consistency-in-replication-96e64f7b5c6 ","permalink":"https://mohibul.pages.dev/posts/posts/java/100daysofjava/day80/","summary":"Implementing a Dummy Consistent Hashing in Java using TreeMap and Hash Function","title":"Day 80: From Concept to Code: Implementing a Dummy Consistent Hashing in Java using TreeMap"},{"content":"In this twitter post https://x.com/ekzhang1/status/1785886484140065087 the author saved $27000/year by removing 10 lines of code they were print() statements. So I just wondered as he did it for python does java printing causes lot of processing time? and if so how about writing the same string in a file. This question let me down a rabbit hole of how printing works in java\nMany developers, especially those new to Java, often rely on System.out.println() for debugging and logging purposes. While this approach is convenient during development, it can become a performance hindrance in production environments, particularly when dealing with high-volume logging scenarios.\nThe Problem with System.out.println() The System.out.println() method is a synchronous operation that involves several steps:\nAcquiring a lock: Before writing to the console, the method must acquire a lock on the System.out object to ensure thread safety. String concatenation: If the argument passed to println() is not a string, it must be converted to a string using the toString() method, which can be a costly operation for complex objects. Writing to the console: The actual writing of the string to the console stream occurs in this step. Flushing the stream: After writing, the console stream is flushed to ensure that the data is immediately visible. Releasing the lock: Finally, the lock on the System.out object is released. While these steps may seem trivial, they can introduce significant overhead, especially when logging is performed extensively in performance-critical sections of the code.\nFurthermore, If we look into why System.out.println() is slow we will find some stackoverflow discussion about it. From one of it\u0026rsquo;s discussion I came to know printing to the console (using println) is slow because of the operating system overhead, not the act of printing itself. They compare it to saving a file, which is similar speed. The slowdown comes from the console displaying characters one by one, including rendering fonts and potentially scrolling the screen.\nFortunately, Java provides more efficient alternatives for logging, including writing to files and using dedicated logging libraries like Log4j.\nWriting to a File Writing log messages directly to a file can be more efficient than printing to the console. Here\u0026rsquo;s an example code snippet:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 import java.io.BufferedWriter; import java.io.FileWriter; import java.io.IOException; public class FileLogger { private static final String LOG_FILE = \u0026#34;application.log\u0026#34;; public static void main(String[] args) { try (BufferedWriter writer = new BufferedWriter(new FileWriter(LOG_FILE, true))) { for (int i = 0; i \u0026lt; 1000000; i++) { writer.write(\u0026#34;Hello World \u0026#34; + i + \u0026#34;\\n\u0026#34;); } } catch (IOException e) { e.printStackTrace(); } } } In this example, we create a BufferedWriter to write log messages to a file named application.log. The BufferedWriter helps improve performance by buffering the data before writing it to the file. However, manually managing file I/O operations can be cumbersome and error-prone, especially in multithreaded environments.\nUsing Log4j with Console Appender Log4j is a widely-used logging library that provides a robust and configurable logging solution. Instead of writing directly to the console or a file, Log4j allows you to configure various appenders (output destinations) and log levels. Here\u0026rsquo;s an example of using Log4j with a console appender:\n1 2 3 4 5 6 7 8 9 10 11 12 import org.apache.logging.log4j.LogManager; import org.apache.logging.log4j.Logger; public class Log4jExample { private static final Logger logger = LogManager.getLogger(Log4jExample.class); public static void main(String[] args) { for (int i = 0; i \u0026lt; 1000000; i++) { logger.info(\u0026#34;Hello World \u0026#34; + i); } } } In this example, we use the LogManager class to obtain a Logger instance for the Log4jExample class. The logger.info() method is used to log messages at the INFO level. Log4j provides various log levels (e.g., DEBUG, INFO, WARN, ERROR) and allows you to configure the desired log level and appenders through configuration files or programmatically.\nLog4j offers several advantages over System.out.println():\nAsynchronous logging: Log4j can buffer log messages and write them asynchronously, reducing the impact on the main application thread. Configurability: Log4j allows you to configure various appenders (e.g., file, console, database), log levels, and formatting options. Performance optimization: Log4j employs techniques like message construction avoidance, which means that log messages are only constructed if they meet the configured log level, reducing unnecessary string operations. Benchmarking with JMH To put these claims to the test, we\u0026rsquo;ll use the Java Microbenchmark Harness (JMH), a powerful tool for benchmarking Java code. Here\u0026rsquo;s an example of how we can benchmark the three techniques discussed above:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 @BenchmarkMode(Mode.AverageTime) @OutputTimeUnit(TimeUnit.MILLISECONDS) @State(Scope.Thread) public class Day79 { private static final Logger logger = LogManager.getLogger(Day79.class); private static final String LOG_FILE = \u0026#34;logfile.txt\u0026#34;; int numIterations = 100000; @Benchmark public void systemOutPrintln() { for (int i = 0; i \u0026lt; numIterations; i++) { System.out.println(\u0026#34;Hello World!\u0026#34;); } } @Benchmark public void writeToFile() throws IOException { try { Path path = Paths.get(LOG_FILE); Files.createFile(path); List\u0026lt;String\u0026gt; lines = new ArrayList\u0026lt;\u0026gt;(Collections.nCopies(numIterations, \u0026#34;Hello World\u0026#34;)); Files.write(path, lines, StandardOpenOption.APPEND); System.out.println(\u0026#34;File written successfully.\u0026#34;); } catch (IOException e) { System.err.println(\u0026#34;Error writing to file: \u0026#34; + e.getMessage()); } } @Benchmark public void log4jConsoleAppender() { for (int i = 0; i \u0026lt; numIterations; i++) { logger.info(\u0026#34;Hello World!\u0026#34;); } } } In this example, we define three benchmark methods: systemOutPrintln(), writeToFile(), and log4jConsoleAppender(). Each method represents one of the logging techniques we want to benchmark.\nThe @BenchmarkMode annotation specifies that we want to measure the average time taken by each method. The @OutputTimeUnit annotation sets the time unit for the benchmark results to nanoseconds. Finally, the @State(Scope.Thread) annotation ensures that each benchmark method is executed in a separate thread.\nAfter running the benchmark, we should see results similar to the following:\nRun complete. Total time: 00:26:52\rBenchmark Mode Cnt Score Error Units\rDay79.log4jConsoleAppender avgt 25 0.121 ± 0.004 ms/op\rDay79.systemOutPrintln avgt 25 4089.977 ± 417.122 ms/op\rDay79.writeToFile avgt 25 0.346 ± 0.071 ms/op The results are given in milliseconds per operation (ms/op), which means how many milliseconds it takes on average to perform one operation. The lower the ms/op, the faster the operation.\nDay79.log4jConsoleAppender: This operation takes on average 0.121 ms/op. This is the fastest operation among the three.\nDay79.systemOutPrintln: This operation takes on average 4089.977 ms/op. This is significantly slower than the other two operations.\nDay79.writeToFile: This operation takes on average 0.346 ms/op. This is slower than log4jConsoleAppender but much faster than systemOutPrintln.\nThe ± value represents the error in the measurement. It means the actual value could be within this range. For example, for writeToFile, the actual average could be between (0.346 - 0.071) and (0.346 + 0.071).\nThese results clearly show that System.out.println() is significantly slower than the Log4j console appender, and writing to a file is the slowest option among the three.\nConclusion In this blog post, we explored the performance implications of using System.out.println() for logging in Java applications. We discussed more efficient alternatives, such as writing to files and using dedicated logging libraries like Log4j. Additionally, we provided benchmark results using JMH to demonstrate the performance differences between these techniques.\nWhile System.out.println() may be convenient for simple debugging during development, it should be avoided in performance-critical scenarios or production environments. Instead, consider using a logging library like Log4j, which offers better performance, configurability, and flexibility.\n","permalink":"https://mohibul.pages.dev/posts/posts/java/100daysofjava/day79/","summary":"Day 79: Like Python In java printing takes too much processing time which costs money","title":"Day 79: Did you know that too much print statement can cost you money"},{"content":"In my previous blog post I talked about Java Unsafe API and how to use it to make a Memory Allocator. Now as Java 22 released new FFM API is here to provide safety that we didnt had while using Java Unsafe API to allocate off the heap memory. In this post I will discuss about the new api and how it works and how we can use this to create safe memory allocator.\nMemorySegment API and Arena in Java FFM The Foreign Function and Memory (FFM) API introduced in Java 20 allows Java programs to interoperate with code and data outside of the Java runtime. This API provides a safer and more controlled way to interact with native code compared to JNI. One of the key components of the FFM API is the MemorySegment API, which provides a Java representation of off-heap memory. Arena, on the other hand, is a memory management construct that controls the lifecycle of memory segments.\nMemorySegment API A MemorySegment object represents a contiguous block of off-heap memory. It provides methods to allocate, deallocate, access, and deallocate memory. Memory segments are language-agnostic, meaning they can be used to hold any type of data, not just Java objects.\nHere are some of the key features of the MemorySegment API:\nAllocation and Deallocation: Memory segments can be allocated in various ways, including fixed-size allocations and allocations that grow or shrink as needed.\nAccess: Memory segments can be accessed using Java byte buffers, which provide a familiar way to work with raw memory.\nDeallocation: Memory segments can be explicitly deallocated when they are no longer needed. This helps to avoid memory leaks.\nArena An Arena is a memory management construct that simplifies the lifecycle management of memory segments. It provides a pool of memory that can be used to allocate memory segments. Arenas can be configured with different memory allocation strategies and deallocation behavior.\nHere are some of the key benefits of using arenas:\nImproved Memory Management: Arenas simplify memory management by providing a centralized pool of memory for allocation.\nReduced Memory Fragmentation: Arenas can help to reduce memory fragmentation by allocating memory segments from a contiguous block.\nThread Safety: Arenas are thread-safe, meaning that multiple threads can safely allocate and deallocate memory segments from the same arena.\nAn arena manages access to native memory and ensures that allocated memory blocks are released again and that we do not access memory that has already been released.\nThere are four types of arenas that we can create using static factory methods of the Arena class:\nGlobal arena Automatic arenas (managed by the garbage collector) Confined arenas Shared arenas Memory Allocator and De-allocator with Arena and MemorySegment Here\u0026rsquo;s an example of a simple memory allocator and de-allocator class that uses an Arena to allocate and deallocate memory segments for writing strings:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 import java.lang.foreign.*; public class MemoryAllocator { private static final Arena arena = Arena.create(); public static MemorySegment allocate(int size) { return arena.allocate(size); } public static void deallocate(MemorySegment segment) { segment.close(); } } now using the MemoryAllocator class\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 public Class Day78 { public static void main(String[] args) { //Initialize MemorySegment segment = allocate(100); //write hello world to the initialized memory segment segment.getByteBuffer(0, 100).put(\u0026#34;Hello, world!\u0026#34;.getBytes()); //read from the memory segment byte[] bytes = new byte[100]; segment.getByteBuffer(0, 100).get(bytes); String message = new String(bytes); System.out.println(message); // deallocate the memorySegment deallocate(segment); } } Here private static final Arena arena = Arena.create(); This line creates a single, static instance of Arena and assigns it to the arena field. An Arena is a memory pool used to allocate memory segments. Since this is declared static and final, only one Arena instance is created for the entire class and its behavior cannot be changed after initialization.\npublic static MemorySegment allocate(int size) This is a public static method named allocate. It takes an integer size as input and returns a MemorySegment object. This method delegates the allocation task to the arena instance by calling arena.allocate(size). It essentially requests a block of off-heap memory of the specified size from the Arena pool.\npublic static void deallocate(MemorySegment segment) This is another public static method named deallocate. It takes a MemorySegment object as input and doesn\u0026rsquo;t return any value (void). This method calls segment.close() to release the memory segment back to the Arena pool. It signifies that the allocated memory is no longer needed.\nFor further learning you can visit this java documentation. https://docs.oracle.com/en/java/javase/20/core/foreign-function-and-memory-api.html#GUID-1CA5C639-6C65-45A7-85E5-7926DB0D6A32\n","permalink":"https://mohibul.pages.dev/posts/posts/java/100daysofjava/day78/","summary":"Day 78: Java 22 - Embracing Safe Memory Allocation Using FFM API - An alternative to Unsafe API","title":"Day 78: Java 22 - Embracing Safe Memory Allocation Using FFM API - An alternative to Unsafe API"},{"content":"In the realm of Java development, the sun.misc.Unsafe API stands as both a powerful tool and a notorious enigma. It offers a direct interface to native memory and low-level operations, granting developers unprecedented control over memory management and object manipulation. While officially designated as \u0026ldquo;unsafe\u0026rdquo; due to its potential for misuse, this API remains indispensable in certain scenarios, enabling performance optimizations and facilitating the implementation of critical functionality in numerous Java libraries and frameworks.\nUnderstanding the Intricacies of Unsafe API At its core, the Unsafe API provides a means to perform operations typically restricted by the Java Virtual Machine (JVM) for safety reasons. These operations include direct memory access, object allocation and manipulation, thread synchronization, and more. By bypassing the usual safeguards of the JVM, developers can achieve performance gains and implement functionalities that would otherwise be impossible or impractical within the confines of standard Java.\nOne of the primary use cases of the Unsafe API is for off-heap memory management. Unlike Java\u0026rsquo;s managed heap, which is subject to garbage collection and automatic memory management, off-heap memory managed via Unsafe offers greater control and reduced overhead. This is particularly beneficial in scenarios requiring predictable memory allocation and deallocation, such as high-performance computing, networking, and database systems.\nsun.misc.Unsafe API in Java provides access to low-level, unsafe operations typically reserved for the JVM. It exposes methods that allow direct manipulation of memory, objects, and threads, bypassing some of Java\u0026rsquo;s built-in safety mechanisms. Here\u0026rsquo;s how it works and what it enables:\nMemory Operations Object Manipulation Array Operations Synchronization Class and Native memory operations Unsafe: Bypassing the Guardrails Java enforces memory safety through garbage collection and automatic memory management. The Unsafe API circumvents these mechanisms, allowing direct manipulation of memory addresses. This capability unlocks several functionalities:\nMemory Allocation and Deallocation: You can allocate off-heap memory (outside the JVM\u0026rsquo;s managed heap) for performance-critical tasks. This is useful for large data structures or handling native data types. Field and Array Manipulation: The Unsafe API allows direct access and modification of fields and array elements, even if they are declared private or final. This can be helpful for performance optimization or implementing custom data structures. Thread Management: Unsafe provides low-level control over thread scheduling and synchronization. This is rarely used in practice due to the complexity and potential for race conditions. Unsafe and Native Memory: A Delicate Dance The Unsafe API interacts with native memory (memory managed by the operating system) through pointers. These pointers hold memory addresses that the Unsafe API can use to:\nAccess Native Data Structures: You can interact with C libraries or legacy code that relies on native data structures. The Unsafe API allows you to convert Java objects to their corresponding native memory representation for seamless data exchange. Memory-Mapped Files: Memory-mapped files allow you to work with a file as if it were part of your program\u0026rsquo;s virtual memory. The Unsafe API facilitates this by creating mappings between file regions and memory addresses. we will dive deep on the Native memory operations by writing a memory allocator and deallocator and using it to write something directly to the memory.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 import sun.misc.Unsafe; public class CustomMemoryAllocator { private static final Unsafe unsafe = Unsafe.getUnsafe(); private static final long BASE_OFFSET = unsafe.arrayBaseOffset(byte[].class); public static long allocateMemory(long bytes) { return unsafe.allocateMemory(bytes); } public static void freeMemory(long address) { unsafe.freeMemory(address); } public static void writeData(long address, byte[] data) { for (int i = 0; i \u0026lt; data.length; i++) { unsafe.putByte(address + i, data[i]); } } public static byte[] readData(long address, int length) { byte[] data = new byte[length]; for (int i = 0; i \u0026lt; length; i++) { data[i] = unsafe.getByte(address + i); } return data; } } Here we are acquiring a static reference to the Unsafe object, the gateway for direct memory operations. - private static final long BASE_OFFSET = unsafe.arrayBaseOffset(byte[].class);: Determines the starting offset of byte arrays within memory, crucial for efficient manipulation.\nThe unsafe.arrayBaseOffset(byte[].class)This is a method call on the unsafe object. The arrayBaseOffset method retrieves the base offset of the given array type. In this case, it\u0026rsquo;s finding the base offset for byte[] arrays.\nThen allocateMemory(long bytes) - Requests a memory block of the specified size from the operating system, bypassing JVM\u0026rsquo;s heap. It returns a memory address representing the allocated block\u0026rsquo;s start.\nThen freeMemory(long address): Deallocates the memory block associated with the given address, relinquishing it back to the OS. Crucial for preventing memory leaks in custom allocation scenarios.\nNow let us use this to write string into an allocated memory and then free the memory\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 public class Day77 { public static void main(String[] args) { long address = CustomMemoryAllocator.allocateMemory(1024); // Allocate 1 KB of memory if (address != 0) { // Ensure memory allocation was successful System.out.println(\u0026#34;Memory allocated successfully at address: \u0026#34; + address); // Write data to the allocated memory CustomMemoryAllocator.writeData(address, \u0026#34;Hello, World!\u0026#34;.getBytes()); // Read data from the allocated memory byte[] data = CustomMemoryAllocator.readData(address, 13); // Read 13 bytes System.out.println(\u0026#34;Data read from memory: \u0026#34; + new String(data)); // Free the allocated memory CustomMemoryAllocator.freeMemory(address); System.out.println(\u0026#34;Memory freed successfully.\u0026#34;); } else { System.err.println(\u0026#34;Failed to allocate memory.\u0026#34;); } } } In the above code we can see that we are storing the data and getting an address and if the memory allocation is successful then we write the data in the given address and then after reading it and printing it we deallocate the memory and clean it. This all are maintained by the run time and garbage collector.\n","permalink":"https://mohibul.pages.dev/posts/posts/java/100daysofjava/day77/","summary":"Explore Something Unsafe : Crafting a Memory allocator using Java Unsafe API","title":"Day 77: Explore Something Unsafe : Crafting a Memory allocator using Java Unsafe API"},{"content":"Service Provider Interface (SPI) that plays a pivotal role in enhancing modularity, extensibility, and maintainability of Java applications. In this blog post, we will delve into the intricacies of SPI, explore its real-world applications, and elucidate why it has become indispensable in modern software development.\nUnderstanding the SPI: The Service Provider Interface is a framework in Java that facilitates the development of service-oriented architectures. At its core, SPI defines a contract between service providers and consumers, allowing for the dynamic loading of implementations without altering the client code. This decoupling of interface and implementation is instrumental in achieving modularity and fosters the creation of pluggable components within a software system.\nAt the heart of SPI is the java.util.ServiceLoader class, which acts as the service provider discovery mechanism. By employing SPI, developers can design extensible and modular systems that can be easily extended or modified without requiring changes to the existing codebase.\nNow let us understand what is Service Oriented Architecture (SOA) means:\nService Oriented Architecture (SOA) Service Oriented Architecture (SOA) is a design approach that structures software applications as a collection of services that are loosely coupled and interoperable. In SOA, services are self-contained, modular units that perform specific business functionalities and communicate with each other over a network. These services can be developed using different programming languages and technologies, as long as they agree upon to a common communication protocol, such as HTTP or SOAP.\nAt the core of SOA is the concept of service autonomy, which means that each service is independent and can be developed, deployed, and scaled independently of other services. This enables greater flexibility and agility in software development, as changes or updates to one service do not require modifications to other services\nExample: Now for a real world example we will create a simple example of a service-oriented architecture using Java Service Provider Interface (SPI). In this example, we\u0026rsquo;ll develop a basic payment processing system with multiple payment gateways. The main application will be the client, and various payment gateways will act as service providers.\n1 2 3 public interface PaymentGateway { void processPayment(double amount); } this will be the service interface.\nNow we will implement two Service Providers for this service interface and those are\nPayPal Service Provider Stripe Service Provider 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 import java.net.HttpURLConnection; import java.net.URL; public class PayPalGateway implements PaymentGateway { private static final String PAYPAL_API_ENDPOINT = \u0026#34;https://api.paypal.com/processPayment\u0026#34;; @Override public void processPayment(double amount) { System.out.println(\u0026#34;Processing payment of $\u0026#34; + amount + \u0026#34; using PayPal Gateway\u0026#34;); try { URL url = new URL(PAYPAL_API_ENDPOINT); HttpURLConnection connection = (HttpURLConnection) url.openConnection(); connection.setRequestMethod(\u0026#34;POST\u0026#34;); // logic int responseCode = connection.getResponseCode(); if (responseCode == HttpURLConnection.HTTP_OK) { System.out.println(\u0026#34;Payment processed successfully with PayPal\u0026#34;); } else { System.out.println(\u0026#34;Payment processing failed with PayPal. Response code: \u0026#34; + responseCode); } } catch (Exception e) { e.printStackTrace(); System.out.println(\u0026#34;Error processing payment with PayPal: \u0026#34; + e.getMessage()); } } } now lets implement the Stripe Payment Service Provider\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 import java.net.HttpURLConnection; import java.net.URL; public class StripeGateway implements PaymentGateway { private static final String STRIPE_API_ENDPOINT = \u0026#34;https://api.stripe.com/processPayment\u0026#34;; @Override public void processPayment(double amount) { System.out.println(\u0026#34;Processing payment of $\u0026#34; + amount + \u0026#34; using Stripe Gateway\u0026#34;); // Call Stripe API to process the payment try { URL url = new URL(STRIPE_API_ENDPOINT); HttpURLConnection connection = (HttpURLConnection) url.openConnection(); connection.setRequestMethod(\u0026#34;POST\u0026#34;); int responseCode = connection.getResponseCode(); if (responseCode == HttpURLConnection.HTTP_OK) { System.out.println(\u0026#34;Payment processed successfully with Stripe\u0026#34;); } else { System.out.println(\u0026#34;Payment processing failed with Stripe. Response code: \u0026#34; + responseCode); } } catch (Exception e) { e.printStackTrace(); System.out.println(\u0026#34;Error processing payment with Stripe: \u0026#34; + e.getMessage()); } } } Now we need to configure the service provider by Service Provider Configuration: META-INF/services/com.example.PaymentGateway\nCreate a file named META-INF/services/com.example.PaymentGateway in your project resources. In this file, specify the fully qualified names of your payment gateway implementations like this:\n1 2 com.example.PayPalGateway com.example.StripeGateway Now in the main class where we will call the Service providers using Service Loader\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 import java.util.ServiceLoader; public class Day76 { public static void main(String[] args) { double amount = 100.0; // Payment amount // Load payment gateways using ServiceLoader ServiceLoader\u0026lt;PaymentGateway\u0026gt; serviceLoader = ServiceLoader.load(PaymentGateway.class); // Process payment using each loaded payment gateway for (PaymentGateway paymentGateway : serviceLoader) { paymentGateway.processPayment(amount); } } } In this example, the PaymentProcessor client application dynamically loads and utilizes different payment gateway implementations without having direct dependencies on them. The service provider configurations allow you to easily extend the system with new payment gateways by adding their implementations and updating the configuration file.\nBut there are some pros and cons of SPI and knowing about those will help us correctly use this API\nThe Java Service Provider Interface (SPI) is a mechanism that facilitates the development of service-oriented architectures by providing a way for services to be implemented and loaded dynamically at runtime. Like any technology, Java SPI comes with its own set of advantages and drawbacks. Let\u0026rsquo;s explore the pros and cons of using Java SPI:\nPros: Dynamic Service Loading: One of the primary advantages of Java SPI is its ability to dynamically load service implementations without modifying the client code. This promotes modularity and allows for the easy addition or removal of services during runtime, fostering flexibility in the application\u0026rsquo;s architecture.\nDecoupling of Components: SPI enables a clear separation between service interfaces and their implementations. This decoupling ensures that changes to service implementations do not impact the client code, promoting a clean and modular design. It facilitates the creation of loosely coupled, pluggable components.\nEasily Extensible: Java SPI provides an easy and standardized way to extend the functionality of an application. New service implementations can be added by creating a new provider class and updating the service provider configuration file without affecting existing code.\nCollaboration and Ecosystem: SPI encourages collaboration among development teams by allowing them to provide and consume services independently. This collaborative nature fosters the creation of a rich ecosystem of interchangeable components and plugins.\nService Discovery: The ServiceLoader class simplifies service discovery, making it straightforward for the application to locate and load available service implementations. This is especially beneficial in large-scale applications with numerous services.\nCons: Limited Compile-Time Checking: Since the service provider and implementation relationship is established at runtime, there is limited compile-time checking for correctness. This can lead to issues such as missing or incorrectly configured service providers, which may only become apparent during runtime.\nReflection Overhead: Java SPI relies on reflection to load service implementations dynamically. Reflection can introduce runtime overhead, potentially affecting performance. However, the impact is generally minimal unless the application involves a large number of service providers.\nGlobal Configuration File: The service provider configuration file (META-INF/services) is a global file that lists all available service implementations. In scenarios with multiple modules or libraries, conflicts may arise if different modules define service providers with the same fully qualified names.\nLimited Metadata: SPI does not provide built-in support for metadata associated with service implementations. Developers might need to rely on conventions or additional mechanisms to convey metadata information about services.\nServiceLoader Limitations: The ServiceLoader API has some limitations, such as a lack of support for specifying the order in which service providers should be loaded. This may be a concern in cases where the order of service providers is crucial.\n","permalink":"https://mohibul.pages.dev/posts/posts/java/100daysofjava/day76/","summary":"Implementing a dummy Service Oriented Architecture (SOA) service to Understand Java Service Provider Interface (SPI)","title":"Day 76: Implementing a dummy Service Oriented Architecture (SOA) service to Understand Java Service Provider Interface (SPI)"},{"content":"In this blog post, I will shine a spotlight on the significance of closing a Java BufferedReader after its creation, exploring the inner workings of BufferedReader, its implications, and offering practical insights to enhance your coding practices.\nUnderstanding BufferedReader: Java\u0026rsquo;s BufferedReader class is a versatile tool for efficient reading of text from a character-based input stream. One of its defining features is the ability to buffer input, enhancing performance by reducing the number of I/O operations. While it offers enhanced reading capabilities, developers must exercise caution, ensuring that the BufferedReader is closed once its purpose is served.\nThe Consequences of Neglecting Closure: Failing to close a BufferedReader can lead to resource leaks and adverse consequences for your application. Java\u0026rsquo;s garbage collector is responsible for reclaiming resources, but it may not be immediate or deterministic. A resource leak can accumulate over time, impacting the overall performance and reliability of your application. Furthermore, in scenarios where the code interacts with external systems or files, leaving the BufferedReader unclosed can lead to file locks and other resource contentions, causing unintended issues.\nPerformance Issues: Memory Usage: An unclosed BufferedReader can lead to increased memory usage over time. The buffer associated with the reader retains resources until the garbage collector decides to reclaim them. In the absence of explicit closure, these resources may persist longer than necessary, contributing to higher memory consumption.\nNow let us see a code example where not closing the BufferedReader can cause high memory usage.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 import java.io.BufferedReader; import java.io.FileReader; import java.io.IOException; public class Day75 { public static void main(String[] args) throws IOException { BufferedReader bufferedReader = new BufferedReader(new FileReader(\u0026#34;sample.txt\u0026#34;)); StringBuilder content = new StringBuilder(); String line; while ((line = bufferedReader.readLine()) != null) { content.append(line).append(System.lineSeparator()); } // Imagine forgetting to close the BufferedReader // bufferedReader.close(); // Uncommenting this line is crucial // Do something with the content, if needed System.out.println(\u0026#34;File Content:\\n\u0026#34; + content.toString()); } } In this example we can see that after reading from the file if we dont close the buffered reader it will keep the reference of the buffered reader from getting garbage collected and thus will have the allocated memory intact, though we dont need the file data and buffered reader instance memory.\nFile Descriptor Exhaustion: Let us first understand what a file descriptor is.\nFile Descriptor : A file descriptor is a low-level integer value that uniquely identifies an open file within a process in a Unix-based operating system. It serves as a handle or reference to an open file or I/O resource, and it is used by the operating system to keep track of various properties and status information associated with the file.\nFile descriptors are represented as non-negative integers. Standard input, output, and error have the file descriptor values 0, 1, and 2, respectively. Additional file descriptors are typically assigned starting from 3 and incrementing for each new open file.\nIn Java, while the concept of file descriptors exists at the operating system level, Java itself abstracts many details through higher-level constructs like streams and readers/writers. When working with Java\u0026rsquo;s I/O classes, you often don\u0026rsquo;t directly deal with file descriptors, but the underlying operating system still manages them behind the scenes\nWhen dealing with file-based input streams, each open file consumes a file descriptor. Failing to close a BufferedReader means leaving file descriptors unclosed. In scenarios where the application handles a large number of files or operates for an extended period, this can result in file descriptor exhaustion. Once the system runs out of available file descriptors, it may fail to open new files, causing unexpected errors and disruptions.\nNow let us see a code example for this\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 import java.io.BufferedReader; import java.io.FileReader; import java.io.IOException; public class Day75 { public static void main(String[] args) { for (int i = 1; i \u0026lt;= 1000; i++) { readFromFile(\u0026#34;file\u0026#34; + i + \u0026#34;.txt\u0026#34;); } } private static void readFromFile(String filePath) { BufferedReader bufferedReader = null; try { bufferedReader = new BufferedReader(new FileReader(filePath)); String line; while ((line = bufferedReader.readLine()) != null) { // Process the content (in this example, we are just printing it) System.out.println(line); } } catch (IOException e) { e.printStackTrace(); } finally { // Missing: Properly close the BufferedReader // Uncomment the following line to avoid file descriptor exhaustion // try { bufferedReader.close(); } catch (IOException e) { e.printStackTrace(); } } } } In this example:\nThe main method iterates over a range of file names (e.g., \u0026ldquo;file1.txt\u0026rdquo;, \u0026ldquo;file2.txt\u0026rdquo;, \u0026hellip;, \u0026ldquo;file1000.txt\u0026rdquo;).\nThe readFromFile method is responsible for reading the content from each file using a BufferedReader.\nThe BufferedReader is not properly closed in the finally block, simulating a scenario where developers forget to close the resource.\nTo see the impact of file descriptor exhaustion, you can run this program in an environment where there is a limit on the number of allowed open file descriptors. After running for some time, the program may encounter issues due to the exhaustion of file descriptors.\nTo avoid file descriptor exhaustion, it\u0026rsquo;s crucial to close resources properly. Uncommenting the bufferedReader.close(); line in the finally block ensures that the file descriptor associated with each BufferedReader is released after reading from the file.\nResource Contention: In cases where the code interacts with external resources, such as databases or network connections, leaving the BufferedReader unclosed can lead to resource contention. This occurs when the resources are not released promptly, and subsequent operations may be delayed or blocked due to the lingering open connections.\nResource contention occurs in computing when multiple processes or threads compete for access to a shared resource, and this competition leads to delays, reduced performance, or potential deadlocks. A shared resource can be any entity that is accessed or modified by multiple concurrent operations, such as files, database connections, network sockets, or even sections of memory.\nNow let us see a code example\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 import java.io.BufferedReader; import java.io.FileReader; import java.io.IOException; public class Day75 { private static final String FILE_PATH = \u0026#34;shared-file.txt\u0026#34;; public static void main(String[] args) { // Create two threads that read from the same file Thread thread1 = new Thread(() -\u0026gt; readFromFile()); Thread thread2 = new Thread(() -\u0026gt; readFromFile()); // Start the threads thread1.start(); thread2.start(); } private static void readFromFile() { BufferedReader bufferedReader = null; try { bufferedReader = new BufferedReader(new FileReader(FILE_PATH)); String line; while ((line = bufferedReader.readLine()) != null) { // Simulate reading from the file (e.g., processing the line) System.out.println(Thread.currentThread().getName() + \u0026#34;: \u0026#34; + line); } } catch (IOException e) { e.printStackTrace(); } finally { // Missing: Properly close the BufferedReader // Uncomment the following line to avoid resource contention // try { bufferedReader.close(); } catch (IOException e) { e.printStackTrace(); } } } } The main method creates two threads (thread1 and thread2) that concurrently attempt to read from the same file (shared-file.txt).\nThe readFromFile method is responsible for reading the content from the file using a BufferedReader.\nThe BufferedReader is not properly closed in the finally block, simulating a scenario where developers forget to close the resource, leading to resource contention.\nTo observe the resource contention, run this program and observe the empty output from both threads. Uncommenting the bufferedReader.close(); line in the finally block ensures that the resource contention is mitigated by properly closing the BufferedReader. Without proper closure, multiple threads may interfere with each other, leading to unpredictable and undesirable outcomes.\nResource Leaks: File Locks: When reading from files, especially in a multi-threaded environment, failing to close a BufferedReader can result in file locks not being released. This is critical because file locks prevent other processes or threads from accessing the file. Over time, accumulating file locks due to unclosed readers can lead to contention and potential deadlock situations.\nIn the above example of Resource contention can generate file locks that will prevent other instances of buffered reader to get the file to read.\nNetwork Resource Leaks: When dealing with network-related input streams, such as reading from a URL, leaving the BufferedReader unclosed can result in resource leaks in the network stack. This may lead to issues such as connection leaks, where network resources are not released promptly, affecting the application\u0026rsquo;s ability to establish new connections.\nNow let us see a code example for this\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 import java.io.BufferedReader; import java.io.IOException; import java.io.InputStreamReader; import java.net.URL; public class NetworkResourceLeakExample { private static final String URL_STRING = \u0026#34;https://www.example.com\u0026#34;; public static void main(String[] args) { // Create two threads that read from the same URL Thread thread1 = new Thread(() -\u0026gt; readFromURL()); Thread thread2 = new Thread(() -\u0026gt; readFromURL()); // Start the threads thread1.start(); thread2.start(); } private static void readFromURL() { BufferedReader bufferedReader = null; try { URL url = new URL(URL_STRING); bufferedReader = new BufferedReader(new InputStreamReader(url.openStream())); String line; while ((line = bufferedReader.readLine()) != null) { // Simulate reading from the URL (e.g., processing the line) System.out.println(Thread.currentThread().getName() + \u0026#34;: \u0026#34; + line); } } catch (IOException e) { e.printStackTrace(); } finally { // Missing: Properly close the BufferedReader // Uncomment the following line to avoid network resource leak // try { bufferedReader.close(); } catch (IOException e) { e.printStackTrace(); } } } } The main method creates two threads (thread1 and thread2) that concurrently attempt to read from the same URL (https://www.example.com).\nThe readFromURL method is responsible for reading the content from the URL using a BufferedReader.\nThe BufferedReader is not properly closed in the finally block, simulating a scenario where developers forget to close the resource, leading to a potential network resource leak.\nTo observe the potential network resource leak, run this program and observe the interleaved output from both threads. Uncommenting the bufferedReader.close(); line in the finally block ensures that the network resource leak is mitigated by properly closing the BufferedReader. Without proper closure, multiple threads may interfere with each other, leading to unpredictable and undesirable outcomes when reading from the shared network resource.\nIn essence, proper closure of the BufferedReader is essential for efficient resource management. By closing the reader explicitly, developers release system resources promptly, preventing memory bloat, file descriptor exhaustion, and resource contention. Failure to do so can lead to a cascade of performance issues and resource leaks that, although subtle, can have a significant impact on the reliability and efficiency of the Java application over time.\n","permalink":"https://mohibul.pages.dev/posts/posts/java/100daysofjava/day75/","summary":"Java Buffered Reader and why we need to close and what will happend if we dont","title":"Day 75: Java BufferedReader: Do We Know Why We Need To Close It?"},{"content":"In the vast realm of Java programming, security is not just a feature; it\u0026rsquo;s a necessity. In this article, we\u0026rsquo;ll embark on a journey to explore the intricate world of Java security by crafting a toy TLS server using the Java Security API. Buckle up as we delve into the nuts and bolts of secure communication!\nSetting the Stage Before we dive into the code, let\u0026rsquo;s set the stage. Understanding the basics of TLS (Transport Layer Security) is crucial. We\u0026rsquo;ll discuss why secure communication is essential and lay the groundwork for our TLS server.\nTLS Transport Layer Security (TLS) is a cryptographic protocol that ensures privacy and data integrity between two communicating applications over a network. It is widely used to secure communication over the internet, particularly in web browsers accessing websites, email communication, and other applications that require secure data transmission.\nHere\u0026rsquo;s an overview of how TLS works:\nHandshake Protocol:\nClientHello: The process begins with the client sending a \u0026ldquo;Hello\u0026rdquo; message to the server, specifying the supported cryptographic algorithms and other parameters. ServerHello: The server responds with its own \u0026ldquo;Hello\u0026rdquo; message, selecting a cryptographic algorithm and other parameters from the client\u0026rsquo;s list. Key Exchange: The server provides its public key to the client, and they agree on a pre-master secret, which will be used to generate encryption keys. Key Exchange:\nThe client encrypts the pre-master secret using the server\u0026rsquo;s public key and sends it back to the server. Both the client and server independently generate a session key from the pre-master secret. Authentication:\nThe server may provide a digital certificate to prove its identity. The client verifies the certificate\u0026rsquo;s authenticity using a trusted Certificate Authority (CA). Optionally, the client may also provide its own certificate for server authentication. Session Key Establishment:\nThe client and server use the agreed-upon session key to generate encryption keys for securing the data transmission. Secure Data Transmission:\nWith the session keys established, the client and server use symmetric encryption to secure the data exchanged during the session. TLS provides confidentiality by encrypting the data, integrity by using Message Authentication Codes (MACs), and authenticity through digital certificates. Finished:\nBoth the client and server send a \u0026ldquo;Finished\u0026rdquo; message to confirm that the handshake is complete. The encrypted session is now established. How TLS uses Key Pair TLS uses key pairs, consisting of a public key and a private key, to secure communications over the internet. These keys serve various purposes during the TLS handshake process:\nAuthentication:\nServer Authentication: The server presents its public key certificate during the handshake to prove its identity. The client can verify the server\u0026rsquo;s authenticity using a trusted Certificate Authority\u0026rsquo;s public key. Client Authentication (Optional): In cases requiring mutual authentication, the client also presents its public key certificate to the server. Key Exchange:\nTLS employs asymmetric key exchange algorithms to establish a shared secret key between the client and server. The public keys in the key pairs facilitate a secure exchange, ensuring that eavesdroppers cannot intercept the shared secret. Data Encryption:\nOnce the shared secret is established, symmetric encryption algorithms are used for efficient data encryption. The security of the initial key exchange, facilitated by the key pairs, is crucial for protecting the confidentiality and integrity of the transmitted data. Perfect Forward Secrecy (PFS):\nTLS emphasizes Perfect Forward Secrecy, ensuring that compromising long-term private keys does not compromise past communications. Key pairs play a role in achieving PFS, especially with algorithms supporting forward secrecy. %% TLS In Java In Java, TLS is typically implemented using the Java Secure Socket Extension (JSSE) framework, which provides the necessary classes for secure communication.\nNow let\u0026rsquo;s see how can we construct and toy TLS server and understand how it works\nCode Crafting Begins Now, the real fun begins. We\u0026rsquo;ll start coding our toy TLS server step by step. From setting up the server socket to handling client connections, we\u0026rsquo;ll navigate the Java Security API to ensure our server stands robust in the face of potential threats.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 import javax.net.ssl.*; import javax.security.cert.CertificateException; import java.io.BufferedReader; import java.io.FileInputStream; import java.io.IOException; import java.io.InputStreamReader; import java.io.PrintWriter; import java.security.KeyManagementException; import java.security.KeyStore; import java.security.KeyStoreException; import java.security.NoSuchAlgorithmException; import java.security.UnrecoverableKeyException; public class Day74 { public static void main(String[] args) { try { SSLContext sslContext = SSLContext.getInstance(\u0026#34;TLS\u0026#34;); KeyManagerFactory keyManagerFactory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm()); KeyStore keyStore = KeyStore.getInstance(\u0026#34;JKS\u0026#34;); char[] password = \u0026#34;javaSecurityDay74\u0026#34;.toCharArray(); // Replace with your keystore password keyStore.load(new FileInputStream(\u0026#34;keystore.jks\u0026#34;), password); // Replace with your keystore file path keyManagerFactory.init(keyStore, password); sslContext.init(keyManagerFactory.getKeyManagers(), null, null); SSLServerSocketFactory sslServerSocketFactory = sslContext.getServerSocketFactory(); SSLServerSocket sslServerSocket = (SSLServerSocket) sslServerSocketFactory.createServerSocket(8888); // Replace with your preferred port System.out.println(\u0026#34;Server started. Waiting for client connection...\u0026#34;); SSLSocket sslSocket = (SSLSocket) sslServerSocket.accept(); BufferedReader in = new BufferedReader(new InputStreamReader(sslSocket.getInputStream())); PrintWriter out = new PrintWriter(sslSocket.getOutputStream(), true); String clientMessage = in.readLine(); System.out.println(\u0026#34;Client message: \u0026#34; + clientMessage); out.println(\u0026#34;Hello, Client!\u0026#34;); in.close(); out.close(); sslSocket.close(); sslServerSocket.close(); } catch (NoSuchAlgorithmException | KeyManagementException | IOException | KeyStoreException | UnrecoverableKeyException | java.security.cert.CertificateException e) { e.printStackTrace(); } } } SSL Context Now lets understand the above code step by step to get understanding how it\u0026rsquo;s communicating using TLS\n1 2 SSLContext sslContext = SSLContext.getInstance(\u0026#34;TLS\u0026#34;); sslContext.init(keyManagerFactory.getKeyManagers(), null, null); here we are instantiating SSLContext which is the main api that\u0026rsquo;s driving the security of the communication. We are requesting the SSLContext with \u0026quot;TLS\u0026quot; and then initializing it with KeyManagers from keyManager factory. Now let us understand what is KeyManagerFactory\nKeyManager and KeyStore In Java, a KeyManager is part of the Java Secure Socket Extension (JSSE) and is responsible for managing key materials for key exchange during SSL/TLS communication. It works in conjunction with TrustManager and SecureRandom to provide a comprehensive security framework for Java applications.\nIn Java, a KeyStore is a repository that stores cryptographic keys and their associated X.509 certificate chains. It is a fundamental component of the Java Cryptography Architecture (JCA) and is used for managing cryptographic keys, certificates, and related information in a secure manner. The KeyStore class is part of the java.security package.\nKeyStores are commonly used in various security-related operations, such as SSL/TLS communication, code signing, and digital signatures.\nNow lets instantiate a KeyManager and then use keystore to store the keypair\n1 2 3 4 5 KeyManagerFactory keyManagerFactory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm()); KeyStore keyStore = KeyStore.getInstance(\u0026#34;JKS\u0026#34;); char[] password = \u0026#34;javaSecurityDay74\u0026#34;.toCharArray(); // Replace with your keystore password keyStore.load(new FileInputStream(\u0026#34;keystore.jks\u0026#34;), password); // Replace with your keystore file path keyManagerFactory.init(keyStore, password); here we are instantiating KeyManagerFactory which manages keyStores which in turns manages keypair. Here we are creating a key manager by pointing it to an already created keystore and then initializing it.\nKeyPair Generation Keys are the backbone of encryption. We\u0026rsquo;ll explore the art of KeyPair generation. To generate key pair we need to create a keystore as keys and certificates are stored inside the keystore.\nTo generate a Java KeyStore (JKS) file, you can use the keytool command-line utility that comes with the Java Development Kit (JDK). Below is an example command to generate a simple self-signed certificate and store it in a JKS file named keystore.jks. This example uses a 2048-bit RSA key for simplicity:\n1 keytool -genkeypair -keyalg RSA -keysize 2048 -alias mykey -keystore keystore.jks -validity 365 -keypass keyPassword -storepass keystorePassword -dname \u0026#34;CN=MyKey, OU=MyOrganizationalUnit, O=MyOrganization, L=MyCity, ST=MyState, C=MyCountry\u0026#34; this command does the following\n-genkeypair: Generates a key pair (public and private keys). -keyalg RSA: Specifies the key algorithm (RSA in this case). -keysize 2048: Specifies the key size (2048 bits in this case). -alias mykey: Specifies the alias for the key pair. -keystore keystore.jks: Specifies the name of the keystore file. -validity 365: Specifies the validity period of the key pair in days. -keypass keyPassword: Specifies the password for the private key. -storepass keystorePassword: Specifies the password for the keystore. -dname \u0026quot;CN=MyKey, OU=MyOrganizationalUnit, O=MyOrganization, L=MyCity, ST=MyState, C=MyCountry\u0026quot;: Specifies the distinguished name (DN) for the certificate. After running this command, you will have a keystore.jks file in your working directory. Remember to replace placeholder values (like passwords and distinguished name) with your own values.\nSSL Server Socket Setup An SSL Server Socket, or SSLServerSocket, is a class in Java that represents a server-side endpoint for SSL/TLS-secured communication. It is part of the Java Secure Socket Extension (JSSE) and is used for establishing secure connections between servers and clients over a network.\nhere we are creating a SSLServerSocketFactory that will provide us with a server socket connection. Then we configure the socket with which port it will listen. In this example I have selected 8888 as the port. The SSLServer will listen on this port. Furthermore we call accept() on SSLServerSocket which will now accept connections.\n1 2 3 4 5 6 SSLServerSocketFactory sslServerSocketFactory = sslContext.getServerSocketFactory(); SSLServerSocket sslServerSocket = (SSLServerSocket) sslServerSocketFactory.createServerSocket(8888); // Replace with your preferred port System.out.println(\u0026#34;Server started. Waiting for client connection...\u0026#34;); SSLSocket sslSocket = (SSLSocket) sslServerSocket.accept(); Input and Output Stream Setup After that we setup input and output stream for incoming and sending data back to client. SSLServerSocket includes an InputStream which alongside with a reader we can get client data and show it\n1 2 3 4 5 6 7 8 9 10 11 12 BufferedReader in = new BufferedReader(new InputStreamReader(sslSocket.getInputStream())); PrintWriter out = new PrintWriter(sslSocket.getOutputStream(), true); String clientMessage = in.readLine(); System.out.println(\u0026#34;Client message: \u0026#34; + clientMessage); out.println(\u0026#34;Hello, Client!\u0026#34;); in.close(); out.close(); sslSocket.close(); sslServerSocket.close(); to learn more about tls https://hpbn.co/transport-layer-security-tls/\n","permalink":"https://mohibul.pages.dev/posts/posts/java/100daysofjava/day74/","summary":"Understanding how Symmetric and Asymmetric encryption works using Java Cryptography Extension","title":"Day 74: Java Security: Crafting a toy TLS Server with Java Security API"},{"content":"The Java Cryptography Extension (JCE) is a framework that provides cryptographic functionality within the Java platform. It offers a set of APIs, algorithms, and services for various cryptographic operations, including encryption, decryption, digital signatures, key management, and more. I will discuss in a 3 part series about the details of\nEncryption and Decryption using Java Digital Signatures KeyManagement and why we need it. So in the first part let us dive into the Encryption using AES algorithm by using some already prebuilt java apis.\nEncryption and Decryption: JCE allows you to perform encryption and decryption using symmetric and asymmetric encryption algorithms. So we should understand brefily about symmetric and asymmetirc encryption.\nSymmetric Encryption Symmetric Encryption is a type of encryption where only one key (a secret key) is used to both encrypt and decrypt electronic data. The entities communicating via symmetric encryption must exchange the key so that it can be used in the decryption process. This encryption method differs from asymmetric encryption where a pair of keys - one public and one private - is used to encrypt and decrypt messages. By using symmetric encryption algorithms, data is \u0026ldquo;scrambled\u0026rdquo; so that it can\u0026rsquo;t be understood by anyone who does not possess the secret key to decrypt it. Once the intended recipient who possesses the key has the message, the algorithm reverses its action so that the message is returned to its original readable form. The secret key that the sender and recipient both use could be a specific password/code or it can be random string of letters or numbers that have been generated by a secure random number generator (RNG).\nCode Example - Symmetric Encryption (AES):\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 import javax.crypto.*; import java.security.Key; import java.security.NoSuchAlgorithmException; public class Day73 { public static void main(String[] args) throws NoSuchAlgorithmException, NoSuchPaddingException, InvalidKeyException, IllegalBlockSizeException, BadPaddingException { // Generate a secret key KeyGenerator keyGenerator = KeyGenerator.getInstance(\u0026#34;AES\u0026#34;); keyGenerator.init(128); SecretKey secretKey = keyGenerator.generateKey(); // Initialize the cipher for encryption Cipher cipher = Cipher.getInstance(\u0026#34;AES\u0026#34;); cipher.init(Cipher.ENCRYPT_MODE, secretKey); String plaintext = \u0026#34;Sensitive information\u0026#34;; byte[] encrypted = cipher.doFinal(plaintext.getBytes()); System.out.println(\u0026#34;Encrypted: \u0026#34; + new String(encrypted)); // Initialize the cipher for decryption cipher.init(Cipher.DECRYPT_MODE, secretKey); byte[] decrypted = cipher.doFinal(encrypted); System.out.println(\u0026#34;Decrypted: \u0026#34; + new String(decrypted)); } } Now in this example we can see some java api\u0026rsquo;s. Specifically KeyGenerator, SecretKey and Cipher.\nJava has a KeyGenerator class : This class is designed to generate secret (symmetric) keys, offering the capabilities of a key generator. You can create KeyGenerator instances using any of the getInstance methods provided by this class.\nKeyGenerator instances can be used multiple times; that is, once a key has been created, the same KeyGenerator can be used to create more keys.\nThere are two methods to create a key: one that is algorithm-independent, and another that is specific to an algorithm. The only distinction between these two methods is how the object is initialized. For the algorithm specific ones the java platform provides some algorithms we can choose from and they are\nAES (128) DES (56) DESede (168) HmacSHA1 HmacSHA256 Cipher class gives us the option to select the algorithms that we want to use to encrypt or decrypt. In our case we have chosen AES as algorithm to encrypt by initializing once as ENCRYPT_MODE and when decrypting as DECRYPT_MODE.\nTo learn more about AES and how it works watch this AES Explained (Advanced Encryption Standard) - Computerphile\nAsymmetric Encryption Asymmetric encryption, also known as public-key cryptography, is a cryptographic system that uses a pair of keys - a public key and a private key - to encrypt and decrypt data. These keys are mathematically related but cannot be derived from one another. The public key is used to encrypt data, while the private key is used to decrypt the data. This system allows for secure communication without the need for the sender and receiver to share the same key beforehand.\nHere\u0026rsquo;s how asymmetric encryption works: Asymmetric encryption, also known as public-key cryptography, is a method of encrypting and decrypting information using a pair of keys: a public key and a private key. It provides a secure way to transmit sensitive data over insecure channels, such as the internet. Here\u0026rsquo;s a simple example for software developers:\nImagine two people, Alice and Bob, who want to communicate securely over an insecure channel. Both Alice and Bob have a pair of keys, one public and one private.\nKey Generation: Alice generates a public-private key pair. She keeps her private key secret and shares her public key with Bob. Similarly, Bob does the same and shares his public key with Alice.\nEncryption: If Alice wants to send a confidential message to Bob, she uses Bob\u0026rsquo;s public key to encrypt the message. This encrypted message can only be decrypted using Bob\u0026rsquo;s private key.\nDecryption: When Bob receives the encrypted message, he uses his private key to decrypt it. Only Bob possesses the private key necessary to unlock the message encrypted with his public key.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 import javax.crypto.Cipher; import java.security.*; import java.util.Base64; public class Day73 { public static void main(String[] args) throws Exception { // Generate Key Pair KeyPair keyPair = generateKeyPair(); System.out.println(keyPair.getPrivate()); System.out.println(keyPair.getPublic()); // Message to be encrypted String originalMessage = \u0026#34;Hello, this is a secret message!\u0026#34;; // Encryption String encryptedMessage = encrypt(originalMessage, keyPair.getPublic()); System.out.println(\u0026#34;Original Message: \u0026#34; + originalMessage); System.out.println(\u0026#34;Encrypted Message: \u0026#34; + encryptedMessage); // Decryption String decryptedMessage = decrypt(encryptedMessage, keyPair.getPrivate()); System.out.println(\u0026#34;Decrypted Message: \u0026#34; + decryptedMessage); } public static KeyPair generateKeyPair() throws NoSuchAlgorithmException { KeyPairGenerator keyPairGenerator = KeyPairGenerator.getInstance(\u0026#34;RSA\u0026#34;); keyPairGenerator.initialize(4096); return keyPairGenerator.generateKeyPair(); } public static String encrypt(String plaintext, PublicKey publicKey) throws Exception { Cipher encryptCipher = Cipher.getInstance(\u0026#34;RSA\u0026#34;); encryptCipher.init(Cipher.ENCRYPT_MODE, publicKey); byte[] cipherText = encryptCipher.doFinal(plaintext.getBytes()); return Base64.getEncoder().encodeToString(cipherText); } public static String decrypt(String encryptedText, PrivateKey privateKey) throws Exception { Cipher decriptCipher = Cipher.getInstance(\u0026#34;RSA\u0026#34;); decriptCipher.init(Cipher.DECRYPT_MODE, privateKey); byte[] decryptedText = decriptCipher.doFinal(Base64.getDecoder().decode(encryptedText)); return new String(decryptedText); } } now lets see how this code does that\nKey Pair Generation: The generateKeyPair() method uses the KeyPairGenerator class to create a new key pair for encryption and decryption. It initializes the key pair generator with the RSA algorithm and a key size of 2048 bits.\nEncryption: The encrypt() method takes a plaintext message and a public key as input. It initializes a Cipher object for encryption using the RSA algorithm. The method then encrypts the plaintext message using the provided public key and returns the encrypted message as a Base64-encoded string.\nDecryption: The decrypt() method takes the encrypted text and the corresponding private key as input. It initializes a Cipher object for decryption using the RSA algorithm. The method then decrypts the Base64-encoded encrypted text using the provided private key and returns the decrypted message as a string.\nIn the main method, a new key pair is generated. The original plaintext message is encrypted using the public key from the generated key pair. The encrypted message is then printed to the console. Next, the encrypted message is decrypted using the private key from the same key pair, and the decrypted message is printed to the console.\nThe example demonstrates how to securely encrypt and decrypt messages using asymmetric encryption with the RSA algorithm in Java. The Base64 class is used to encode and decode byte arrays as strings to ensure that the encrypted and decrypted messages can be properly handled as strings.\n","permalink":"https://mohibul.pages.dev/posts/posts/java/100daysofjava/day73/","summary":"Understanding how Symmetric and Asymmetric encryption works using Java Cryptography Extension","title":"Day 73: Java Security: Safeguarding Data with Encryption and Keys"},{"content":"While working with a problem I came upon an intriguing question. Can a HashMap Take Null or Empty String as a Key in Java?. If possible how does it translate to the serialization and how popular libraries will handle this situation.\nBefore starting let us understand how does HashMap works in java. Sepcifically we will focus on the part of how it creates a hash for the hashmap.\nWhat is a Hashmap At the core of HashMap is the concept of hashing. When you put a key-value pair into a HashMap, the key is first hashed to determine its storage location within the underlying array. The hash code is an integer value computed from the key\u0026rsquo;s contents using the key\u0026rsquo;s hashCode() method.\nNow a question arises. Do null have a hashcode?\nLets see\n1 System.out.println(Objects.hashCode(null)); here I am using the native Objects class which havea hashCode(). It can create hash for the given object. In this case this method will return 0 for null.\nIn jdk source code for HashMap class one can see that below method is implemented with the hashCode() method.\n1 2 3 public final int hashCode() { return Objects.hashCode(key) ^ Objects.hashCode(value); } this method calculates the hash code for a key-value pair by combining the hash codes of both the key and the value using the bitwise XOR (^) operation. The Objects.hashCode() method is used to obtain the hash codes for the key and value, and then they are XORed together to produce a single hash code.\nSo its returning a valid integer that will point to the bucket where the value will be stored. In this case what is a bucket? A bucket is a position in the underlying array, and multiple key-value pairs with the same hash code are stored in the same bucket as a linked list (known as a collision chain).\nNow let us understand a little bit more about how this hash and storing works.\nHashMap Internal Array Internally, a HashMap uses an array to store its elements. The size of this array is initially set to a default capacity, but it dynamically resizes and rehashes its elements as necessary to maintain efficient performance.\nWhen a key-value pair is added to the HashMap, the key\u0026rsquo;s hash code is used to compute an index in the array. If the bucket at that index is empty, the key-value pair is stored there. If there is already a key-value pair(s) in the bucket due to a hash code collision, the new key-value pair is linked to the existing ones as a part of the linked list.\nHash Collision Handling Hash collisions occur when two different keys have the same hash code. In such cases, multiple key-value pairs are stored in the same bucket as a linked list. To find the correct value associated with a key during retrieval, the HashMap traverses the linked list (collision chain) until it finds the key or reaches the end of the chain.\nTo improve performance and minimize the impact of hash collisions, it\u0026rsquo;s crucial to have a good hash function that evenly distributes hash codes across the array. The ideal hash function produces distinct hash codes for distinct keys while avoiding excessive collisions.\nSo back to the question from this we can understand that null can be a valid key for an Hashmap\nAlso its mention in the documenation of JDK. Now lets see can empty string can be a valid key.\nAs an empty string still and String object in java so it will generate a valid hashcode when we do\n1 System.out.println(\u0026#34;\u0026#34;.hashCode()); Similar to null it will generate a valid hashcode. By the way if you run both \u0026quot;\u0026quot; and null in Objects.hashCode() it will return 0 as int. From the code of Objects.hashCode() we can see why.\n1 2 3 public static int hashCode(Object o) { return o != null ? o.hashCode() : 0; } Also from the hashCode implementation of string we can see\n1 2 3 4 5 6 7 8 9 10 11 12 public int hashCode() { int h = hash; // hash by default 0 if (h == 0 \u0026amp;\u0026amp; value.length \u0026gt; 0) { char val[] = value; for (int i = 0; i \u0026lt; value.length; i++) { h = 31 * h + val[i]; } hash = h; } return h; } when the string is empty then it will return 0.\n","permalink":"https://mohibul.pages.dev/posts/posts/java/100daysofjava/day72/","summary":"Can a HashMap Take Null as a Key in Java?","title":"Day 72: Can a HashMap Take Null or Empty String as a Key in Java?"},{"content":"Java is a statically typed language that provides type safety to ensure that variables are used consistently and correctly. However, there are cases where type safety can be compromised, leading to unexpected behavior and runtime errors. One such scenario is heap pollution. In this blog post, we will explore the concept of Java heap pollution, understand why it happens, learn how to detect it, and discuss potential solutions to mitigate its risks.\nWhat is Java Heap Pollution? Heap pollution occurs in Java when a variable of a parameterized type references an object of an incompatible type. It results in the introduction of elements into a parameterized type structure that violate its type constraints, compromising type safety.\nWhy Does Heap Pollution Happen? Heap pollution can happen due to several reasons:\nIncompatible Assignments: Heap pollution can occur when assigning objects of incompatible types to variables of parameterized types. For example, assigning a List to a List variable.\nLet\u0026rsquo;s say you have a method that accepts a List parameter and attempts to add elements from a source list to it. However, if the source list is a List, heap pollution occurs. Here\u0026rsquo;s an example:\n1 2 3 4 5 6 7 8 9 10 public static void addElementsToList(List\u0026lt;Object\u0026gt; destinationList, List\u0026lt;?\u0026gt; sourceList) { for (Object obj : sourceList) { destinationList.add(obj); // Heap pollution occurs here } } // Usage List\u0026lt;String\u0026gt; stringList = Arrays.asList(\u0026#34;Hello\u0026#34;, \u0026#34;World\u0026#34;); List\u0026lt;Object\u0026gt; objectList = new ArrayList\u0026lt;\u0026gt;(); addElementsToList(objectList, stringList); // Heap pollution occurs In this case, the attempt to add String objects to a List leads to heap pollution. The type safety is compromised, and unexpected behavior can occur when accessing elements from the objectList.\nUnchecked Type Conversions: Improper unchecked type conversions can introduce heap pollution. When casting objects to parameterized types without proper type checks, the wrong types can be assigned, leading to heap pollution. Consider the following example:\n1 2 3 4 5 6 7 8 public static \u0026lt;T\u0026gt; T cast(Object obj) { return (T) obj; // Heap pollution occurs here } // Usage List\u0026lt;Integer\u0026gt; integerList = new ArrayList\u0026lt;\u0026gt;(); List rawList = integerList; List\u0026lt;String\u0026gt; stringList = cast(rawList); // Heap pollution occurs In this case, the cast method attempts to perform an unchecked type conversion of a raw list to a list of strings. This unchecked conversion introduces heap pollution, as the list contains integer elements but is cast to a string list, potentially causing type safety violations and unexpected behavior.\nVarargs and Generics: Varargs methods combined with generics can also introduce heap pollution. When invoking varargs methods with incompatible types, heap pollution can occur. Consider the following example:\n1 2 3 4 5 6 7 8 9 10 11 @SafeVarargs public static \u0026lt;T\u0026gt; List\u0026lt;T\u0026gt; createList(T... elements) { List\u0026lt;T\u0026gt; list = new ArrayList\u0026lt;\u0026gt;(); for (T element : elements) { list.add(element); // Heap pollution occurs here } return list; } // Usage List\u0026lt;String\u0026gt; stringList = createList(\u0026#34;Hello\u0026#34;, \u0026#34;World\u0026#34;); Although the createList method is annotated with @SafeVarargs, indicating it is safe from heap pollution, it can still be misused. If the method is invoked with incompatible types, such as createList(\u0026quot;Hello\u0026quot;, 42), heap pollution occurs, leading to potential runtime errors when accessing the elements of the list.\nDetecting Heap Pollution: Detecting heap pollution can be challenging since it often leads to runtime errors rather than compilation errors. However, there are some approaches to detect heap pollution:\nCompiler Warnings: Enable compiler warnings using the -Xlint:unchecked flag. This provides warnings about potential heap pollution issues during compilation, giving you a chance to review and address them.\nStatic Code Analysis Tools: Utilize static code analysis tools that can detect potential heap pollution issues. These tools analyze your code and provide suggestions for improvements to prevent heap pollution scenarios.\nSolving Heap Pollution: To mitigate the risks associated with heap pollution, consider the following solutions:\nUse Parameterized Types Correctly: Ensure that parameterized types are used consistently and correctly throughout your code. Avoid situations where variables of parameterized types reference objects of incompatible types.\nEnable Compiler Warnings and Annotations: Enable compiler warnings to get notified about potential heap pollution issues during compilation. Use the @SuppressWarnings(\u0026ldquo;unchecked\u0026rdquo;) annotation to suppress warnings for well-understood and intentional heap pollution cases.\nValidate and Filter Input: Validate and filter input data to ensure type compatibility and prevent incompatible objects from entering a parameterized type structure. Perform necessary type checks before performing type conversions or assignments.\nDesign Patterns and Best Practices: Apply design patterns, such as the Factory pattern or Builder pattern, to encapsulate object creation and ensure type safety. Follow best practices for generic programming to promote type safety and avoid heap pollution scenarios.\nTesting and Code Reviews: Thoroughly test your code, including scenarios that involve parameterized types, to uncover potential heap pollution issues. Conduct code reviews to identify and address any heap pollution concerns.\nJava heap pollution can introduce type safety violations and unexpected behavior in your code. By understanding the causes, detecting heap pollution, and applying appropriate solutions, you can mitigate the risks and ensure type safety. It is essential to use parameterized types correctly, enable compiler warnings, validate input, follow best practices, conduct thorough testing, and engage in code reviews to prevent heap pollution scenarios.\nBy prioritizing type safety and being vigilant about potential heap pollution, you can write more robust and reliable Java code.\n","permalink":"https://mohibul.pages.dev/posts/posts/java/100daysofjava/day71/","summary":"Understanding Heap poullion in java and its causes and how to prevent it","title":"Day 71: Understanding Heap poullion in java and its causes and how to prevent it"},{"content":"In software engineering, optimizing the performance of an application is crucial for delivering a high-quality user experience. One area where performance optimizations can make a significant impact is in the manipulation of arrays. Efficient array copying operations can lead to faster execution times and better application performance.\nThis is where the Java Microbenchmark Harness (JMH) library comes in handy. JMH is a benchmarking tool designed specifically for measuring the performance of Java code. In this article, we will use JMH to measure the performance of various array copying methods.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 public class ArrayCopyBenchmarks { @State(Scope.Benchmark) public static class ArrState { public int[] arr = new int[3000]; public int[] arr2 = new int[3000]; @Setup public void setup() { Random random = new Random(); Arrays.fill(arr, random.nextInt()); } } //@Benchmark public int[] loopCopy(ArrState state) { int[] arr2 = new int[30]; for (int i = 0; i \u0026lt; state.arr.length; i++) { arr2[i] = state.arr[i]; } return arr2; } //@Benchmark public int[] systemArraysCopy(ArrState state) { int[] arr2 = new int[30]; System.arraycopy(state.arr, 0, arr2, 0, state.arr.length); return arr2; } @Benchmark public int[] systemArraysCopyToExistingArray(ArrState state) { System.arraycopy(state.arr, 0, state.arr2, 0, state.arr.length); return state.arr2; } @Benchmark public int[] loopCopyToExistingArray(ArrState state) { for (int i = 0; i \u0026lt; state.arr.length; i++) { state.arr2[i] = state.arr[i]; } return state.arr2; } //@Benchmark public int[] arraysCopy(ArrState state) { return Arrays.copyOf(state.arr, state.arr.length); } public static void main(String[] args) throws Exception { runBenchmark(ArrayCopyBenchmarks.class); } } Let\u0026rsquo;s start by taking a look at the Java code provided above. The code defines a class called ArrayCopyBenchmarks which contains several methods for copying arrays. These methods include:\nloopCopy: This method uses a simple loop to copy the contents of one array to another. systemArraysCopy: This method uses the System.arraycopy method to copy the contents of one array to another. systemArraysCopyToExistingArray: This method uses the System.arraycopy method to copy the contents of one array to an existing array. loopCopyToExistingArray: This method uses a simple loop to copy the contents of one array to an existing array. arraysCopy: This method uses the Arrays.copyOf method to copy the contents of one array to another. Each of these methods is annotated with the @Benchmark annotation, which tells JMH to measure their execution time.\nThe main method of the ArrayCopyBenchmarks class uses the runBenchmark method to run the benchmarks and display the results. When you run the program, JMH will execute each benchmark method several times and report the average execution time.\nNow let\u0026rsquo;s look at the results of the benchmarks. We can see that the systemArraysCopy and systemArraysCopyToExistingArray methods are the fastest, followed by the arraysCopy method. The loopCopy and loopCopyToExistingArray methods are the slowest.\nThese results tell us that the System.arraycopy method is the most efficient way to copy arrays in Java. It is faster than using a loop or the Arrays.copyOf method. Additionally, the systemArraysCopyToExistingArray method is faster than creating a new array and copying the contents of the old array to it.\nIn conclusion, optimizing array copying operations can have a significant impact on application performance. By using JMH to measure the performance of different array copying methods, we can determine which method is the most efficient. In this case, the System.arraycopy method is the clear winner. By incorporating this method into your code, you can improve the performance of your Java applications.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day70/","summary":"Measuring the performance of different array copying methods in Java using JMH","title":"Day 70:Diving deep into Array copy- Measuring Array Copy Performance in Java with JMH: A Comparative Analysis of Methods"},{"content":"Java Microbenchmark Harness (JMH) is a Java library used to measure the performance of Java code snippets, methods, and classes. It is a useful tool for developers who want to optimize their Java applications, identify performance bottlenecks, and compare the performance of different implementations.\nJMH provides a simple API for developers to write benchmarks and test their performance. In this blog post, we will explore the basics of JMH and how to write and run benchmarks using JMH.\nWhat is a Benchmark Harness: A benchmark harness is a software that provides the infrastructure to conduct benchmarks of a software and/or hardware system, typically with the goal to quantitatively assess the system\u0026rsquo;s characteristics and capabilities or to compare the characteristics and capabilities of multiple systems relative to each other\nWhat is JMH: JMH is built on top of the Java Virtual Machine\u0026rsquo;s (JVM) HotSpot compiler infrastructure. It uses the Just-In-Time (JIT) compiler to optimize code at runtime, which means that the code is executed multiple times before the actual benchmark is run. This ensures that the benchmark results are accurate and consistent.\nTo achieve this, JMH generates a benchmarking harness that consists of a loop that executes the benchmark method multiple times. This loop is called a \u0026ldquo;measurement iteration\u0026rdquo; and it is executed several times to ensure that the benchmark results are reliable.\nJMH Execution Lifecyle we\u0026rsquo;ll explore the different phases of the JMH execution lifecycle and what happens during each phase.\nFig: JMH\nSetup Phase The first phase of the JMH execution lifecycle is the setup phase. During this phase, JMH initializes the benchmark state and sets up the environment for the benchmark. The @Setup annotation can be used to define a method that will be executed during this phase.\nWarmup Phase The next phase is the warmup phase. During this phase, JMH runs the benchmark method multiple times to allow the JVM to warm up and optimize the code. The @Warmup annotation can be used to control the duration of the warmup phase and the number of iterations to run.\nMeasurement Phase The measurement phase is the main benchmarking phase. During this phase, JMH measures the performance of the benchmark method. The @Benchmark annotation is used to mark the method to be benchmarked. JMH runs the benchmark method for a configurable number of iterations and measures the execution time of each iteration.\nIteration Tear Down Phase After each iteration of the benchmark method, JMH executes any methods annotated with @TearDownIteration. This can be used to clean up any resources or reset the state of the benchmark.\nResult Aggregation Phase After all iterations have completed, JMH aggregates the results and computes statistics such as the mean, standard deviation, and confidence intervals. This provides a summary of the benchmark results.\nBenchmark Tear Down Phase Finally, JMH executes any methods annotated with @TearDown after the benchmark has completed. This can be used to release any resources or perform any necessary cleanup.\nSetting up JMH Before we dive into writing benchmarks, we need to set up JMH in our project. JMH can be easily added to your project by adding the following dependency to your Maven or Gradle build file:\n1 2 3 4 5 6 7 8 9 \u0026lt;!-- Maven dependency --\u0026gt; \u0026lt;dependency\u0026gt; \u0026lt;groupId\u0026gt;org.openjdk.jmh\u0026lt;/groupId\u0026gt; \u0026lt;artifactId\u0026gt;jmh-core\u0026lt;/artifactId\u0026gt; \u0026lt;version\u0026gt;1.34\u0026lt;/version\u0026gt; \u0026lt;/dependency\u0026gt; \u0026lt;!-- Gradle dependency --\u0026gt; compile \u0026#39;org.openjdk.jmh:jmh-core:1.34\u0026#39; Writing a simple benchmark To write a benchmark using JMH, we need to create a class and annotate it with @BenchmarkMode, @OutputTimeUnit, and @State annotations. The @BenchmarkMode annotation specifies the benchmark mode, the @OutputTimeUnit annotation specifies the time unit to use for reporting, and the @State annotation specifies the state of the benchmark.\nHere is an example of a simple benchmark that measures the performance of adding two integers:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 import org.openjdk.jmh.annotations.Benchmark; import org.openjdk.jmh.annotations.BenchmarkMode; import org.openjdk.jmh.annotations.Mode; import org.openjdk.jmh.annotations.OutputTimeUnit; import org.openjdk.jmh.annotations.State; import org.openjdk.jmh.annotations.Scope; import java.util.concurrent.TimeUnit; @State(Scope.Thread) @BenchmarkMode(Mode.AverageTime) @OutputTimeUnit(TimeUnit.NANOSECONDS) public class AddBenchmark { private int a = 1; private int b = 2; @Benchmark public int add() { return a + b; } } In this benchmark, we have annotated the class with @State(Scope.Thread) to indicate that the state of the benchmark is local to each thread, @BenchmarkMode(Mode.AverageTime) to indicate that we want to measure the average time taken by the benchmark, and @OutputTimeUnit(TimeUnit.NANOSECONDS) to indicate that we want to report the results in nanoseconds.\nThe benchmark method is annotated with @Benchmark, which indicates that this is the method that we want to measure. In this case, we are measuring the time taken to add two integers.\nRunning a benchmark To run a benchmark, we can create an instance of the org.openjdk.jmh.runner.Runner class and pass it the class that contains the benchmark. Here is an example of how to run the AddBenchmark class we just created:\n1 2 3 4 5 6 7 8 9 10 11 12 13 import org.openjdk.jmh.runner.Runner; import org.openjdk.jmh.runner.RunnerException; import org.openjdk.jmh.runner.options.Options; import org.openjdk.jmh.runner.options.OptionsBuilder; public class Main { public static void main(String[] args) throws RunnerException { Options options = new OptionsBuilder() .include(AddBenchmark.class.getSimpleName()) .build(); new Runner(options).run(); } } In this example, we have created an instance of the OptionsBuilder class and used it to specify the benchmark class that we want to run. We then create an instance of the Runner class and pass it the options we just created.\nFor further learning Jenkov has a great tutorial about it. https://jenkov.com/tutorials/java-performance/jmh.html\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day69/","summary":"Exploring benchmarking harness in Java and how its use to understand performance","title":"Day 69 : Unlocking Java Performance Secrets: Harnessing the Power of JMH Microbenchmarking"},{"content":"Analyzing for memory leaks in Java involves identifying objects that are no longer being used by the program but are still being held in memory. which can cause memory usage to increase over time and potentially lead to out-of-memory errors. Here are the steps for analyzing and identifying memory leaks in Java:\nIdentify the symptoms: The first step in analyzing for memory leaks is to identify the symptoms of the issue, such as out-of-memory errors or increasing memory usage over time.\nGather data: Once the symptoms have been identified, gather data on the application\u0026rsquo;s memory usage. This can be done using tools such as Java\u0026rsquo;s built-in JMX monitoring, VisualVM or JConsole. JFR is another tool to get events and data that is occuring in the jvm. Java Mission Control provides these data. There are some other tools in this page mentioned below\nhttps://docs.oracle.com/javase/9/troubleshoot/diagnostic-tools.htm#JSTGD117\nCreate heap dumps: At first lets learn what is heap dump.\nHeap dumps : The term Heapdump describes the JVM mechanism that generates a dump of all the live objects that are on the Java™ heap, which are being used by the running Java application. There are two dump formats, the text or classic heap dump format and the Portable Heap Dump (PHD) format.\nCreate heap dumps at regular intervals or when the application experiences an out-of-memory error. Heap dumps can be analyzed using tools like Eclipse Memory Analyzer (MAT) or VisualVM.\nFig 1: VisualVM\nAnalyze heap dumps Analyze the heap dumps to identify the objects that are being held in memory but are no longer being used by the application. MAT and VisualVM can help identify the root cause of memory leaks. Below is the snap for MAT where a heap dump file is loaded\nJava heap dumps can be analyzed using various tools, such as Eclipse Memory Analyzer (MAT) and VisualVM.\nFig 2: Eclipse MAT\nHere are the general steps for analyzing Java heap dumps:\nCapture the heap dump: Heap dumps can be captured using command-line tools like jmap or via the application itself (if it supports such functionality). Heap dumps can also be automatically captured when the application experiences an OutOfMemoryError. Load the heap dump into the analysis tool: After capturing the heap dump, load it into an analysis tool like Eclipse Memory Analyzer (MAT) or VisualVM. Both tools have built-in heap dump analysis capabilities. Analyze the heap dump: Use the analysis tool to explore the heap dump and identify potential memory leaks. The analysis tool will provide information about the objects in the heap, their sizes, and the relationships between them. Identify the root cause: Use the analysis tool to identify the root cause of the memory leak. This may involve identifying objects that are not being garbage collected, objects that are unnecessarily retained in memory, or objects that are being excessively created. Take action: Once the root cause has been identified, take action to address the issue. This may involve modifying the code to properly release resources, updating third-party libraries, or tuning the JVM settings to better manage memory. Verify the fix: After making the necessary changes, verify that the memory leak has been fixed and the application is functioning correctly. Overall, analyzing Java heap dumps requires a deep understanding of the application\u0026rsquo;s memory usage patterns and the tools used for heap dump analysis. By following these general steps, developers can identify and address memory leaks in their Java applications.\nIdentify the cause: Once the objects causing the memory leaks have been identified, determine the root cause of the issue. This may involve reviewing the code to identify incorrect memory management practices or third-party libraries causing the issue.\nFix the issue: Fix the issue by correcting the root cause. This may involve modifying the code to properly release resources, updating third-party libraries, or tuning the JVM settings to better manage memory.\nTest: Test the application to ensure that the memory leak has been fixed and that the application is functioning correctly.\nBy following these steps, developers can identify and address memory leaks in their Java applications. Regular monitoring and analysis can help prevent memory leaks from becoming a significant issue in production environments.\nAnalyze for finding Memory leaks In order analyze whether your program contains any potential Memory Leaks you will need some kind specialized tools like\nHeapHero JProfiler VisualVM these allow you view what exactly happening under hood during runtime \u0026amp; identify problematic areas ahead time before problems start manifesting themselves on production environment.\nIn the case I was working, we started looking into which classes were taking much memory and thread execution as well. Then exported a heap dump and analyzed it with Eclipse MAT and isolated the class which was an old version of a library.\nSteps to prevent Memory leaks To prevent Memory Leaks occurring its important ensure all resources get closed properly at end each operation ; try avoid creating too many temporary variables unnecessarily \u0026amp; keep track object lifetime create them only necessary basis then dispose off quickly once done with it ; finally make sure Garbage Collector running correctly so old unused objects get cleared up regularly thus freeing up valuable system resources\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day68/","summary":"Understanding and finding the causes and types of memory leaks and tools and suggestions to fix them","title":"Day 68 : Becoming a memory plumber; A tale of Memory Leak and how to find them ( part 3)"},{"content":"In my previous post I briefly discussed about few types of memory leaks. In todays post going to further discuss about six other scenarios which can cause memory leaks. They are\nUnclosed resource custom equals and hashcode implementation Inner class that references outer classes Finalization bug Thread local Unclosed Resources Unclosed resource memory leak in Java occurs when an application fails to release resources such as file handles, sockets, and database connections after they are no longer needed. These unclosed resources can remain allocated for extended periods of time resulting in a gradual buildup of system memory. This type of leak is particularly dangerous because it often goes unnoticed until the system runs out of available RAM or disk space.\n1 2 3 4 5 6 7 8 9 10 public void getSaleResults() { try { URL url = new URL(\u0026#34;http://api.com/saleResutlts\u0026#34;); URLConnection urlConnection = url.openConnection(); InputStream is = urlConnection.getInputStream(); byte[] bytes = is.readAllBytes(); } catch (IOException ioe) { ioe.printStackTrace(); } } so to resolve this we should use try-with-resource\nJava developers must be aware that any code which opens external resources such as files or databases should ensure that these are properly closed once their use has concluded. Failure to do so will result in a slow but steady increase in the amount of used memory on the system over time which can eventually lead to performance issues or even complete failure due to lack of available RAM and/or disk space on the machine hosting your application\u0026rsquo;s process(es).\nCustom .equals() and .hashCode() implementation Custom .equals() and .hashCode() implementation can cause memory leaks in Java due to the way they are used. When a class implements these methods, it is responsible for managing its own state and ensuring that all objects of the same type have unique references. If this is not done properly, an object can be created but never garbage collected because it will always exist as a reference from another object or collection. This leads to more objects staying in memory than necessary which causes a gradual increase in system resources over time until eventually there is no longer enough available for other tasks leading to poor performance or even crashing of the application.\nLet\u0026rsquo;s see an example scenario, In HashSet and HashMap .equals() and .hashCode()uses these methods in many operations, and if they\u0026rsquo;re not overridden correctly, they can become a source for potential memory leak problems.\n1 2 3 4 5 6 7 8 9 class SaleResult { int saleCount; boolean isSuccess; public SaleResult(int saleCount, boolean isSuccess) { this.saleCount = saleCount; this.isSuccess = isSuccess; } } now if we set this object as key in a hashmap or hashset and if a string id for value of the cusomterId we would need to ensure they are unique as HashMap and HashSet dont allow duplicate keys.\n1 2 3 4 5 Map\u0026lt;SaleResult, Long\u0026gt; map = new HashMap\u0026lt;\u0026gt;(); for(int i=0; i\u0026lt;10000; i++) { map.put(new Person(\u0026#34;jon\u0026#34;), 1); } System.out.println(map.size()); here we will see 1000 objects are being inserted and thus the size of the map is 1000. Here we\u0026rsquo;re using SaleReuslt as a key. Since Map doesn\u0026rsquo;t allow duplicate keys, the numerous duplicate SaleResult objects that we inserted as a key shouldn\u0026rsquo;t increase the memory.\nBut since we haven\u0026rsquo;t defined the proper equals() method, the duplicate objects pile up and increase the memory, which is why we see more than one object in the memory.\nHowever, if we\u0026rsquo;d overridden the equals() and hashCode() methods properly, then only one Person object would exist in this Map. Let\u0026rsquo;s take a look at the proper implementations of equals() and hashCode() for our SaleResult class:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 class SaleResult { int id; boolean isSuccess; public SaleResult(int id, boolean isSuccess) { this.id = id; this.isSuccess = isSuccess; } @Override public boolean equals(Object o) { if (o == this) return true; if (!(o instanceof SaleResult)) { return false; } SaleResult result = (SaleResult) o; return result.id == this.id; } @Override public int hashCode() { int hash = 7; hash = 31 * hash + id; return hash; } } now after this there will be only 1 instance of SaleResult in the hashmap after insert.\nInner class that references outer classes This happens in the case of non-static inner classes (anonymous classes). For initialization, these inner classes always require an instance of the enclosing class. This type of leak occurs when the inner class holds a reference to the outer class. which prevents the garbage collector from reclaiming any memory associated with outer instance. This causes significant performance degradation and even application crashes if not addressed properly.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 class SaleResult { long saleCount; public SaleResult(long saleCount) { this.saleCount = saleCount; } class SaleData { } SaleData getSaleData() { return new SaleData(); } } now if we run this program like this\n1 2 3 4 5 6 7 8 9 public class Day67 { public static void main(String args[]) { List\u0026lt;SaleResult.SaleData\u0026gt; saleResults = new ArrayList\u0026lt;\u0026gt;(); while(infiniteLoopFlag) { // for analyzing the program saleResults.add(new SaleResult(10).getSaleData()): } } } the inner class SaleData will keep the reference of the Outer class SaleResult thus causing memory not to be GC.\nInner classes are useful for organizing code better, but it is important to consider their impact on memory management when using them in your program design. The best way to prevent this type of issue is through proper usage of weak references within the inner class implementation or using static classes. so that it does not maintain a strong reference back up into its parent or containing object graph structure while still providing access as needed at runtime.\nUsing finalization Sometimes using finalize() can cause memory leaks. Whenever a classes finalize() method is overridden, then objects of that class aren\u0026rsquo;t instantly garbage collected. Instead, the GC queues them for finalization, which occurs at a later point in time.\nAdditionally, if the code written in the finalize() method isn\u0026rsquo;t optimal, and if the finalizer queue can\u0026rsquo;t keep up with the Java garbage collector, then sooner or later our application is destined to meet an OutOfMemoryError.\nThreadLocal ThreadLocal is a api that gives us the ability to store state to a particular thread, and thus allows us to achieve thread safety.\nWhen using this construct, each thread will hold an implicit reference to its copy of a ThreadLocal variable and will maintain its own copy, instead of sharing the resource across multiple threads, as long as the thread is alive.\nDespite its advantages, the use of ThreadLocal variables is controversial, as they\u0026rsquo;re infamous for introducing memory leaks if not used properly. Joshua Bloch once commented on thread local usage that:\n“Can you cause unintended object retention with thread locals? Sure you can. But you can do this with arrays too. That doesn’t mean that thread locals (or arrays) are bad things. Merely that you have to use them with some care. The use of thread pools demands extreme care. Sloppy use of thread pools in combination with sloppy use of thread locals can cause unintended object retention, as has been noted in many places. But placing the blame on thread locals is unwarranted.” – Joshua Bloch\nThreadLocals are supposed to be garbage collected once the holding thread is no longer alive. But the problem arises when we use ThreadLocals along with modern application servers.\nModern application servers use a pool of threads to process requests, instead of creating new ones (for example, the Executor in the case of Apache Tomcat). Moreover, they also use a separate classloader. Since ThreadPools in application servers work on the concept of thread reuse, they\u0026rsquo;re never garbage collected; instead, they\u0026rsquo;re reused to serve another request.\nIf any class creates a ThreadLocal variable, but doesn\u0026rsquo;t explicitly remove it, then a copy of that object will remain with the worker Thread even after the web application is stopped, thus preventing the object from being garbage collected.\nHow does ThreadLocal Creates memory leak Each Thread has a private field threadLocals, which actually stores the thread-local values. Each key in this map is a weak reference to a ThreadLocal object, so after that ThreadLocal object is garbage-collected, its entry is removed from the map. But each value is a strong reference, so when a value (directly or indirectly) points to the ThreadLocal object that is its key, that object will neither be garbage-collected nor removed from the map as long as the thread lives. In this example, the chain of strong references looks like this:\nThread object → threadLocals map → instance of example class → example class → static ThreadLocal field → ThreadLocal object.\nLets see an example of this\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 public class Day67 { private static final ThreadLocal\u0026lt;List\u0026lt;String\u0026gt;\u0026gt; threadLocal = new ThreadLocal\u0026lt;\u0026gt;(); public static void main(String[] args) throws InterruptedException { List\u0026lt;Thread\u0026gt; threads = new ArrayList\u0026lt;\u0026gt;(); for (int i = 0; i \u0026lt; 10; i++) { Thread thread = new Thread(() -\u0026gt; { List\u0026lt;String\u0026gt; list = new ArrayList\u0026lt;\u0026gt;(); threadLocal.set(list); // add a lot of data to the thread-local list for (int j = 0; j \u0026lt; 100000; j++) { list.add(\u0026#34;This is a string that will consume some memory.\u0026#34;); } }); threads.add(thread); thread.start(); } for (Thread thread : threads) { thread.join(); } // At this point, all threads have completed execution. // However, the thread-local lists have not been cleared, // and will continue to consume memory until the program terminates. } } In this program, we define a ThreadLocal variable called threadLocal which stores a list of strings. We then create 10 threads and in each thread, we create a new list and add a lot of data to it. We then set this list as the value of the thread-local variable using threadLocal.set(list).\nBecause the ThreadLocal variable is thread-local, each thread has its own list and the lists are not shared between threads. However, since we are adding a lot of data to each list, they consume a significant amount of memory.\nThe problem is that once each thread has completed execution, the lists are not cleared. Because the ThreadLocal variable still holds a reference to each list, they will not be garbage collected and will continue to consume memory until the program terminates. This can lead to a memory leak, as the program\u0026rsquo;s memory usage will continue to grow over time.\nTo avoid this, we should always make sure to clear the thread-local variables once we are done with them, by calling threadLocal.remove() at the end of each thread\u0026rsquo;s execution.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day67/","summary":"Understanding and finding the causes and types of memory leaks and tools and suggestions to fix them","title":"Day 67: Becoming a memory plumber; A tale of Memory Leak and how to find them ( part 2)"},{"content":"What a Java memory leak is The garbage collector reclaims objects that are no longer reachable. It cannot reclaim an object while application code still holds a reference to it, even when the application no longer needs the object.\nA Java memory leak is therefore a reachability problem. Retained objects accumulate, heap use grows across collection cycles, and the application eventually spends more time collecting garbage or runs out of memory. Heap dumps, VisualVM, Eclipse MAT, and JMX help identify which objects remain reachable and which references retain them.\nAs a software engineer, it’s important to understand the concept of memory leaks in Java. A memory leak is when an application continues to hold onto memory that it no longer needs and this can lead to performance issues or even system crashes. In Java, a common cause of such leaks is objects not being garbage collected as expected due to references still pointing at them from other parts of the code.\nWhen an object is created in Java, the JVM (Java Virtual Machine) allocates space for it on the heap and then assigns a reference variable which points towards that allocated space. As long as there are any active references pointing towards this object, it will remain alive on heap until all these references are removed and GC (Garbage Collector) reclaims its resources back from Heap Memory Pool. If somehow some reference(s) remains active even after they have been used up by program logic then those objects become eligible for Garbage Collection but never get collected since their respective reference variables still point at them causing unnecessary accumulation of unused/unrequired Objects over time resulting into what we call “Memory Leak” issue with our Application Codebase leading into Performance Degradation \u0026amp; System Crashes eventually!\nTo avoid such problems one should keep track if their code has any potential chances where Object References might be left dangling without getting released explicitly using null assignments or try-catch blocks etc., so that GC can reclaim those resources back once they become eligible otherwise Memory Leaks would continue accumulating over time leading us into trouble!\nWhat problems it causes Memory leaks in Java can be caused by a variety of factors such as incorrect garbage collection, resource leakage in APIs, unclosed streams or connections, poor coding practices etc. These issues can lead to decreased responsiveness from applications due to increased latency times; they may also result in OutOfMemoryError exceptions being thrown which causes applications to crash unexpectedly or become unstable over time due their inability handle large amounts of data efficiently anymore .\nTypes of Memory Leaks There are many ways memory leaks can occur in a java program. Some of them are\nStatic fields Loitering objects In memory cache not evicted Unclosed resource custom equals and hashcode implementation Inner class that references outer classes Finalization bug Interned strings Thread local now let us understand these typical memory leak issues\nstatic fields heavy use of static variables can cause memory leaks. In Java, static fields have a lifetime that usually matches the entire lifetime of the running program / application (unless ClassLoader becomes eligible for garbage collection).\nlets see a case\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 public class Day65 { public static List\u0026lt;String\u0026gt; strings = new ArrayList\u0026lt;\u0026gt;(); public void add() { for (int i = 0; i \u0026lt; 100000; i++) { list.add(new String()); } System.out.println(\u0026#34;check point add\u0026#34;); } public static void main(String[] args) { System.out.println(\u0026#34;Beforee calling add\u0026#34;); new Day65().populateList(); System.out.println(\u0026#34;After calling add\u0026#34;); } } here when observing with\nLoitering Objects These objects are allocated memory, but not used, and not garbage collected. These keep increasing the size of the JVM heap and represent memory leaks, which can cause an out-of-memory error or excessive overhead on the garbage collector.\nLet\u0026rsquo;s understand this scenario with an example, consider a stack backed by an array, and you implement pop this way:\n1 2 3 4 5 6 7 public T pop() { if (top == -1) { throw new NoSuchElementException(); } T item = items[top--]; return item; } Because the array holds a reference to the item at the former top of the stack, the item is still strongly reachable from the array and therefore won\u0026rsquo;t be garbage collected. Furthermore, because of the way a stack works, the item is no longer externally accessible. Thus, the item is said to be \u0026ldquo;loitering\u0026rdquo;. It hangs around for no reason.\nThe correct implementation of the pop method would be this:\n1 2 3 4 5 6 7 8 9 10 public T pop() { if (top == -1) { throw new NoSuchElementException(); } T item = items[top]; // Clear the reference to allow garbage collection items[top] = null; top--; return item; } By setting the array entry to null, the item is no longer reachable from the array. Therefore it is eligible for garbage collection.\nGarbage collection isn\u0026rsquo;t magic. It can do most of the work to clean up memory for you. But, you have to make sure you write your code correctly to eliminate unnecessary strong references. The garbage collector can\u0026rsquo;t read your mind, and figure out that you didn\u0026rsquo;t really mean to keep the strong reference.\nIn Memory Cache Eviction when we build in memory cache (in this case Guava library) there is a possibility of memory leak when the cache get increased without clearing the cache. let us first understand what is cache eviction\nCache Eviction Caches are small because cache memory is more expensive to produce and purchase. The small size of the cache limits the amount of data we can store. We need to consider what data we keep or remove over time. This dilemma is the core purpose behind cache eviction. It will depending on algorithms will remove some entries.\nSo by restricting the size or creating an eviction policy will ensure the cache doesn\u0026rsquo;t grow uncontrollably.\nIn the below example we will see a cache without a size or eviction policy\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 public class CustomCache\u0026lt;T\u0026gt; { private Cache\u0026lt;String, T\u0026gt; cache; //Constructor to build Cache Store public CustomCache() { cache = CacheBuilder.newBuilder().build(); } public T get(String key) { return cache.getIfPresent(key); } //Method to put a new record in Cache Store with record key public void add(String key, T value) { if(key != null \u0026amp;\u0026amp; value != null) { cache.put(key, value); System.out.println(\u0026#34;Entry stored in \u0026#34; + value.getClass().getSimpleName() + \u0026#34; Cache with Key = \u0026#34; + key); } } } Now adding a size and cache eviction policy will help not to cause memory leak. In guava CacheBuilder by adding the properties maximumSize and expireAfterAccess will limit the size and when will be the entires will be deleted\n1 2 3 4 5 6 public CustomCache() { cache = CacheBuilder.newBuilder() .maximumSize(100) .expireAfterAccess(30, TimeUnit.MINUTES) .build(); } Part 2 covers other causes of retained memory.\nLimitations The cache example is intentionally small. It demonstrates one retention pattern, but it is not a production incident report. Heap growth can also come from expected load, large live data sets, native allocations, class loaders, threads, or an undersized heap. Confirm the retained path and object growth before changing code.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day66/","summary":"How retained references create Java memory leaks, how heap tools expose them, and how cache bounds prevent one common failure mode.","title":"Finding a Java memory leak with heap dumps and JMX"},{"content":"Remote Method Invocation (RMI) and Remote Procedure Call (RPC) are two important concepts in Java that allow for distributed computing. Both RMI and RPC enable software components to communicate with each other over a network, allowing code to be executed on different machines without the need for manual intervention. In this blog post, we will discuss what RMI and RPC are, how they differ from one another, their advantages/disadvantages as well as an example of each in Java.\nWhat is RMI At its core, Remote Method Invocation is a way of calling methods on remote objects over a network connection using the Object Request Broker Architecture (ORB). It provides developers with the ability to execute code remotely without having any knowledge about where or how it is being run – all you need is an interface definition file (.idl file), which describes your object\u0026rsquo;s methods signatures and return types along with some additional information related to security etc., then you can use it just like any other local object! The main benefit here lies in its flexibility; since there’s no requirement for manual coding or compilation when making changes across multiple systems simultaneously - instead all that needs doing is updating one centralised IDL file.\nRemote invocation is nothing new. For many years C programmers have used remote procedure calls (RPC) to execute a C function on a remote host and return the results. The primary difference between RPC and RMI is that RPC, being an offshoot of the C language, is primarily concerned with data structures. It’s relatively easy to pack up data and ship it around, but for Java, that’s not enough. In Java we don’t just work with data structures; we work with objects, which contain both data and methods for operating on the data. Not only do we have to be able to ship the state of an object (the data) over the wire, but also the recipient has to be able to interact with the object (use its methods) after receiving it.\nNow let us first understand some concepts regarding RMI and how it works. At first lets understad what are Remote and Non Remote Objects\nRemote and Non Remote Obj Before an object can be used with RMI, it must be serializable. But that’s not sufficient. Remote objects in RMI are real distributed objects. As the name suggests, a remote object can be an object on a different machine; it can also be an object on the local host. The term remote means that the object is used through a special kind of object reference that can be passed over the network. Like normal Java objects, remote objects are passed by reference. Regardless of where the reference is used, the method invocation occurs at the original object, which still lives on its original host. If a remote host returns a reference to one of its objects to you. Then you call that object’s methods; the actual method invocations will happen on the remote host, where the object is.\nNon remote objects are simpler. They are just normal serializable objects. (You can pass these over the network). The catch is that when you pass a non remote object over the network it is simply copied. So references to the object on one host are not the same as those on the remote host. Non remote objects are passed by copy (as opposed to by reference).\nStub Stub is used in the implementation of remote objects. When you invoke a method on a remote object (which could be on a different host), you are actually calling some local code that serves as a proxy for that object. This is the stub. (It is called a stub because it is something like a truncated placeholder for the object.)\nThe stub object on the client machine builds an information block and sends this information to the server.\nThe block consists of\nAn identifier of the remote object to be used Method name which is to be invoked Parameters to the remote JVM Skeleton The skeleton is another proxy that lives with the real object on its original host. It receives remote method invocations from the stub and passes them to the object.\nThe skeleton object passes the request from the stub object to the remote object. It performs the following tasks\nIt calls the desired method on the real object present on the server. It forwards the parameters received from the stub object to the method. Before creatin stubs and skeletons we need to create remote object and understand how it works\nRemote Object The first thing to do is to create an interface that will provide the description of the methods that can be invoked by remote clients. This interface should extend the Remote interface and the method prototype within the interface should throw the RemoteException.\nRemote objects are objects that implement a special remote interface that specifies which of the object’s methods can be invoked remotely. The remote interface must extend the java.rmi.Remote interface. Your remote object will implement its remote interface; as will the stub object that is automatically generated for it. In the rest of your code, you should then refer to the remote object as an instance of the remote interface—not as an instance of its actual class. Because both the real object and stub implement the remote interface, they are equivalent as far as we are concerned (for method invocation); locally, we never have to worry about whether we have a reference to a stub or to an actual object. This “type equivalence” means that we can use normal language features, like casting with remote objects. Of course public fields (variables) of the remote object are not accessible through an interface, so you must make accessor methods if you want to manipulate the remote object’s fields.\nAll methods in the remote interface must declare that they can throw the exception java.rmi.RemoteException . This exception (actually, one of many subclasses to RemoteException) is thrown when any kind of networking error happens: for example, the server could crash, the network could fail, or you could be requesting an object that for some reason isn’t available.\nHere’s a simple example of the remote interface that defines the behavior of RemoteObject; we’ll give it a method that can be invoked remotely\n1 2 3 public interface MessageService extends Remote { String send(String message) throws RemoteException; } when we implement this object its called remote object which is for the server. For the client, the RMI library will dynamically create an implementation Stub.\nimplementation\n1 2 3 4 5 6 7 public class MessengerServiceImpl implements MessengerService { @Override public String send(String message) { return \u0026#34;Client Message\u0026#34;.equals(message) ? \u0026#34;Server Message\u0026#34; : null; } } It\u0026rsquo;d be unusual for our remote object to throw a RemoteException since this exception is typically reserved for the RMI library to raise communication errors to the client. Leaving it out also has the benefit of keeping our implementation RMI-agnostic. Also, any additional methods defined in the remote object, but not in the interface, remain invisible for the client.\nOnce we create the remote implementation, we need to bind the remote object to an RMI registry.\nRMI Registry The registry is the RMI phone book. You use the registry to look up a reference to a registered remote object on another host. We’ve already described how remote references can be passed back and forth by remote method calls. But the registry is needed to bootstrap the process: the client needs some way of looking up some initial object.\nThe registry is implemented by a class called Naming and an application called rmiregistry . This application must be running on the local host before you start a Java program that uses the registry. You can then create instances of remote objects and bind them to particular names in the registry. (Remote objects that bind themselves to the registry sometimes provide a main( ) method for this purpose.) A registry name can be anything you choose; it takes the form of a slash-separated path. When a client object wants to find your object, it constructs a special URL with the rmi: protocol, the hostname, and the object name. On the client, the RMI Naming class then talks to the registry and returns the remote object reference.\nSo before registering in RMI registry we need to create a stub\n1 2 3 MessengerService server = new MessengerServiceImpl(); MessengerService stub = (MessengerService) UnicastRemoteObject .exportObject((MessengerService) server, 0); We use the static UnicastRemoteObject.exportObject method to create our stub implementation. The stub communicates with the server over the underlying RMI protocol.\nThe first argument to exportObject is the remote server object. The second argument is the port that exportObject uses for exporting the remote object to the registry.\nGiving a value of zero indicates that we don\u0026rsquo;t care which port exportObject uses, which is typical and so chosen dynamically.\nUnicastRemoteObject The actual implementation of a remote object (not the interface we discussed previously) will usually extend java.rmi.server.UnicastRemoteObject. This is the RMI equivalent to the familiar Object class. When a subclass of UnicastRemoteObject is constructed, the RMI runtime system automatically “exports” it to start listening for network connections from remote interfaces (stubs) for the object. Like java.lang.Object, this superclass also provides implementations of equals( ) , hashcode( ), and toString( ) that make sense for a remote object.\nRMI Architecture Now lets create a RMI registry\nCreating RMI Registry We can start up a registry local to our server or as a separate stand-alone service. Here we will create for local\n1 Registry rmiRegistry = LocateRegistry.createRegistry(1099); This creates a registry to which stubs can be bound by servers and discovered by clients. Also, we\u0026rsquo;ve used the createRegistry method, since we are creating the registry local to the server.\nBy default, an RMI registry runs on port 1099. Rather, a different port can also be specified in the createRegistry factory method.\nBut in the stand-alone case, we\u0026rsquo;d call getRegistry, passing the hostname and port number as parameters.\nNow we have to bind the stub which is generated by rmic to registry\nBind Stub An RMI registry is a naming facility like JNDI etc. We can follow a similar pattern here, binding our stub to a unique key: here its MessageService\n1 registry.rebind(\u0026#34;MessengerService\u0026#34;, stub); after registering the stub in RMI registry we need to create a client.\nCreate Client To do this, we\u0026rsquo;ll first locate the RMI registry. In addition, we\u0026rsquo;ll look up the remote object stub using the bounded unique key.\nAnd finally, we\u0026rsquo;ll invoke the send method:\n1 2 3 4 5 6 7 8 9 10 11 Registry registry = LocateRegistry.getRegistry(); MessengerService server = (MessengerService) registry .lookup(\u0026#34;MessengerService\u0026#34;); // looking for the binded stub String responseMessage = server.send(\u0026#34;Client Message\u0026#34;); String expectedMessage = \u0026#34;Server Message\u0026#34;; if (expectedMessage.equals(responseMessage)) { System.out.println(\u0026#34;Connection Successful\u0026#34;); } else { System.out.println(\u0026#34;Not able to recognize client\u0026#34;); } Because we\u0026rsquo;re running the RMI registry on the local machine and default port 1099, we don\u0026rsquo;t pass any parameters to getRegistry. Indeed, if the registry is rather on a different host or different port, we can supply these parameters. Once we lookup the stub object using the registry, we can invoke the methods on the remote server.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day65/","summary":"Understanding how RPC and RMI works in java","title":"Day 65: A tale of RMI (Remote Method Invocation)  in java"},{"content":"First of all lets understand what is a Bloom Filter and how does it work and what are its usages.\nBloom Filter : Bloom filter is a data structure designed to tell you, rapidly and memory-efficiently, whether an element is present in a set.\nThe price paid for this efficiency is that a Bloom filter is a probabilistic data structure: it tells us that the element either definitely is not in the set or may be in the set\nFor example, checking availability of username. In this case its a set membership problem, where the set is the list of all registered username. The price we pay for efficiency is that it is probabilistic in nature. That means, there might be some False Positive results. False positive means, it might tell that given username is already taken but actually it’s not.\nhere in this scenario we are searching for the user name Mary in the bloom filter. So from the picture we can see a bloom filter is a combination of a hash function and a bit array or list. Where after hashing the user name will give us a number in which we will set the bit to true.\nSo if we use different hash functions and then update the bit array with 1 where the index of the array would be number provided by the hash function. Then if we search the name again with those hash functions and check the index\u0026rsquo;s again and if we find 1 there then we can plausibly say that, that name exist in the bloom filter.\nSo, in a nutshell: If we search for a value and see any of the hashed indexes for this value is ‘0’ then, the value is definitely not on the list. If all of the hashed indexes is ‘1’ then ‘maybe’ the searched value is on the list. Properties : unlike a standard hash table, a Bloom filter of a fixed size can represent a set with large number of elements. False positive rate increases steadily as elements are added until all bits in the filter are set to 1, at which point all queries yield a positive result. So let us discuss what is false positive in the case of getting information from a bloom filter.\nThe question is why we said “Probabilistic”, why this uncertainty? Let’s understand this with an example. Suppose we want to check whether “Mohibul” is present or not. We’ll calculate hashes using h1, h2 and h3 functions\n1 2 3 4 String name = \u0026#34;Mohibul\u0026#34;; h1(name) % 10 = 5; h2(name) % 10 = 2; h3(name) % 10 = 4; If we check the bit array, bits at these indices are set to 1.\nNow lets check for another name \u0026ldquo;Mary\u0026rdquo;\n1 2 3 4 String name = \u0026#34;Mary\u0026#34; h1(name) % 10 = 1; h2(name) % 10 = 2; h3(name) % 10 = 5; but we know that “Mary” was never added to the filter. Bit at index 2 and 5 was set when we added “Mohibul” . And if another name made 1 bit true then this bloom filter will say that \u0026ldquo;Mary\u0026rdquo; exists in the filter but in real we didnt add Mary.\nThis behavior causes false positives.\nBloom filters never generate false negative result, i.e., telling you that a username doesn’t exist when it actually exists. Deleting elements from filter is not possible. Because, if we delete a single element by clearing bits at indices generated by k hash functions, it might cause deletion of few other elements. Example – if we delete “Mohibul” (in given example) by clearing bit at 4, 5 and 2, we might end up deleting “Mary” also if its present Because bit at index 2 and 5 becomes 0 and bloom filter claims that “ Mary ” is not present. Let us make a bloom filter ourselves.\nBefore doing that let us ask some question that will help us better design and understand\nWhat are these hash functions? How big should I make the bloom filters? How many hash functions should i use? Benefits of bloom filter? Where can I use them? What is Hash functions The hash functions used in a Bloom filter should be independent and uniformly distributed. They should also be as fast as possible (cryptographic hashes such as sha1, though widely used thus are not very good choices).\nExamples of fast, simple hashes that are independent enough includes murmur, xxHash, Fowler–Noll–Vo hash function and many others\nHow big should I make the bloom filters It\u0026rsquo;s a nice property of Bloom filters that you can modify the false positive rate of your filter. A larger filter will have less false positives, and a smaller one more.\nYour false positive rate (P) will be approximately\n(1-e^((-kn)/m)))^k so you can just plug the number n of elements you expect to insert, and try various values of k and m to configure your filter for your application.\nHere\nm - is the size of the bit array\rn - is the number of elements\rk - is the number hash funtions This leads to an obvious question\nHow many hash functions should I use? The more hash functions you have, the slower your bloom filter, and the quicker it fills up. If you have too few, however, you may suffer too many false positives.\nSince you have to pick k when you create the filter, you\u0026rsquo;ll have to ballpark what range you expect n to be in. Once you have that, you still have to choose a potential m (the number of bits) and k (the number of hash functions).\nSo now we can design a process to build a bloom filter\nChoose a approx value for n Choose a value for m Calculate the optimal value of k Calculate the error rate for our chosen values of n, m, and k. If it\u0026rsquo;s unacceptable, return to step 2 and change m; otherwise we\u0026rsquo;re done. Now for the code we will use Java\u0026rsquo;s BitSet API\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 public class BloomFilter\u0026lt;T\u0026gt; { private BitSet bitSet; private int n; private int m; private List\u0026lt;Function\u0026lt;T, Integer\u0026gt;\u0026gt; hashFunctions; public BloomFilter(final int n, final int m, final List\u0026lt;Function\u0026lt;T, Integer\u0026gt;\u0026gt; hashFunctions) { this.n = n; this.bitSet = new BitSet(n); this.m = hashFunctions.size(); this.hashFunctions = hashFunctions; } public void add(T value) { for (Function\u0026lt;T, Integer\u0026gt; function : hashFunctions) { int hash = mapHashToInt(function.apply(value)); bitSet.set(hash, true); } } public boolean mightContain(T value) { for (Function\u0026lt;T, Integer\u0026gt; function : hashFunctions) { int hash = mapHashToInt(function.apply(value)); if (!bitSet.get(hash)) { return false; } } return true; } private int mapHashToInt(int hash) { return hash \u0026amp; (n - 1); } } here Function\u0026lt;T,Integer\u0026gt; hashFunctions supplied by the client. And BitSet exactly works like a bit array.\nNow we have tried our custom BloomFilter Google guava has a implementation of BloomFilter which can be created like\n1 2 3 4 BloomFilter\u0026lt;Integer\u0026gt; filter = BloomFilter.create( Funnels.integerFunnel(), 500, 0.01); Benifits of Bloom Filter Given a Bloom filter with m bits and k hashing functions, both insertion and membership testing are O(k). That is, each time you want to add an element to the set or check set membership, you just need to run the element through the k hash functions and add it to the set or check those bits.\nThe space advantages are more difficult to sum up; again it depends on the error rate you\u0026rsquo;re willing to tolerate. It also depends on the potential range of the elements to be inserted; if it is very limited, a deterministic bit vector can do better. If you can\u0026rsquo;t even ballpark estimate the number of elements to be inserted, you may be better off with a hash table or a scalable Bloom filter.\nWhere Can I Use It Bloom filter is all about testing Membership in a set. The classic example of using bloom filters is to reduce expensive disk (or network) lookups for non-existent keys. As we can see that bloom filters can search for a key in O(k) constant time, where k is the number of hash functions, it will be very fast to test non-existence of a key.\nIf the element is not in the bloom filter, then we know for sure we don’t need to perform the expensive lookup. On the other hand, if it is in the bloom filter, we perform the lookup, and we can expect it to fail some proportion of the time (the false positive rate).\nFor some more concrete examples:\nYou’ve seen in our given example that we could’ve use it to warn the user for weak passwords. You can use bloom filter to prevent your users from accessing malicious sites. Instead of making a query to an SQL database to check if a user with a certain email exists, you could first use a bloom filter for an inexpensive lookup check. If the email doesn’t exist, great! If it does exist, you might have to make an extra query to the database. You can do the same to search for if a ‘Username is already taken’. You can keep a bloom filter based on the IP address of the visitors to your website to check if a user to your website is a ‘returning user’ or a ‘new user’. Some false positive value for ‘returning user’ won’t hurt you, right? You can also make a Spell-checker by using bloom filter to track the dictionary words. Want to know how Medium used bloom filter to decide if a user already read post? Read this mind-blowing, freaking awesome article about it. in this wikipidia link there are multiple other usage of bloom filter.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day64/","summary":"Implementing a basic bloom filter to understand its concepts and usages using Java BitSet api","title":"Day 64: Implementing a basic Bloom Filter Using Java BitSet api"},{"content":"What is OutOfMemory java.lang.OutOfMemoryError exception is thrown when there is insufficient space to allocate an object in the Java heap. In this case, The garbage collector cannot make space available to accommodate a new object, and the heap cannot be expanded further. Also, this error may be thrown when there is insufficient native memory to support the loading of a Java class. In a rare instance, a java.lang.OutOfMemoryError may be thrown when an excessive amount of time is being spent doing garbage collection and little memory is being freed.\nWhen a java.lang.OutOfMemoryError exception is thrown, a stack trace is also printed. In that stack trace the cause are mentioned for easier fixing of the issue.\nThis exception will be thrown when\nMemory Leak occurs Not enough heap space to allocate objects If GC is running all the time Trying to allocate memory larger than the heap size When Metaspace memory is full When native memory is not enough to allocate How OOM happens There are many more reasons OOM can occur. An OutOfMemoryError (OOME) is bad. It can happen at any time, with any thread. There is little that you can do about it, except to exit the program, change the -Xmx value, and restart the JVM. If you then make the -Xmx value too large, you slow down your application. The secret is to make the maximum heap value the right size, neither too small, nor too big. OOME can happen with any thread, and when it does, that thread typically stops. Often, there is not enough memory to build up a stack trace for the OOME, so you cannot even determine where it occurred, or why.\nNow lets dive into the alert system code.\n1 2 3 public interface Listener { void alertMemoryLow(long used, long max, Map\u0026lt;Thread, StackTraceElement[]\u0026gt; allThreadStackTrace); } Let us define a Listener which will listen for the alert. Then\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 public static class OOMAlertService { private final List\u0026lt;Listener\u0026gt; listeners = new ArrayList\u0026lt;\u0026gt;(); public OOMAlertService() { MemoryMXBean mbean = ManagementFactory.getMemoryMXBean(); NotificationEmitter emitter = (NotificationEmitter) mbean; emitter.addNotificationListener((notification, o) -\u0026gt; { if (notification.getType().equals( MemoryNotificationInfo.MEMORY_THRESHOLD_EXCEEDED)) { long maxMemory = tenuredGenPool.getUsage().getMax(); long usedMemory = tenuredGenPool.getUsage().getUsed(); for (Listener listener : listeners) { listener.alertMemoryLow(usedMemory, maxMemory, Thread.getAllStackTrace()); } } }, null, null); } public boolean addListener(Listener listener) { return listeners.add(listener); } public boolean removeListener(Listener listener) { return listeners.remove(listener); } private static final MemoryPoolMXBean tenuredGenPool = findTenuredGenPool(); public static void setUsageThreshold(double threshold) { if (threshold \u0026lt;= 0.0 || threshold \u0026gt; 1.0) { throw new IllegalArgumentException(\u0026#34;Threshold Percentage outside range\u0026#34;); } long maxMemory = tenuredGenPool.getUsage().getMax(); long warningThreshold = (long) (maxMemory * threshold); tenuredGenPool.setUsageThreshold(warningThreshold); } private static MemoryPoolMXBean findTenuredGenPool() { for (MemoryPoolMXBean pool : ManagementFactory.getMemoryPoolMXBeans()) { if (pool.getName().contains(\u0026#34;Old\u0026#34;) \u0026amp;\u0026amp; pool.getType() == MemoryType.HEAP \u0026amp;\u0026amp; pool.isUsageThresholdSupported()) { return pool; } } throw new Exception(\u0026#34;Could not find tenured space\u0026#34;); } } In our OOMAlterService we add listeners to implement the OOMAlertService.Listener interface, with one method alertMemoryLow(long used, long max) that will be called when the threshold is reached.\nOnce we have downcast the MemoryMXBean to a NotificationEmitter we can add a NotificationListener to the MemoryMXBean. You should verify that the notification is of type MEMORY_THRESHOLD_EXCEEDED.\nThis notification will be emitted fast.Something to note is that the listener is being called by a special thread, called the Low Memory Detector thread, that is part of the standard JVM.\nWhat is the threshold? And which of the many pools should we monitor? The only sensible pool to monitor is the Tenured Generation (Old Space). When you set the size of the memory with -Xmx256m, you are setting the maximum memory to be used in the Tenured Generation. Searching in findTenuredGenPool() method, and returning the first one that was of type HEAP to get the Tenured gen memory.\nIn setUsageThreshold(doublethreshold) method, I specify when I would like to be notified. This threshold is a global setting. so only one usage threshold per Java Virtual Machine. The threshold value is used to calculate the usage threshold, based on the maximum memory size of the Tenured Generation pool (not the Runtime.getRuntime().maxMemory() value!).\nIn findTenuredGenPool() I am trying to find the Tenured / Old gen memory which is a heap memory. At first lets understand about JVM Heap and Non Heap memory.\nHeap memory The heap memory is the runtime data area from which the Java VM allocates memory for all class instances and arrays. The heap may be of a fixed or variable size. The garbage collector is an automatic memory management system that reclaims heap memory for objects.\nEden Space: The pool from which memory is initially allocated for most objects. Survivor Space: The pool containing objects that have survived the garbage collection of the Eden space. Tenured Generation or Old Gen: The pool containing objects that have existed for some time in the survivor space. Non-heap memory Non-heap memory includes a method area shared among all threads and memory required for the internal processing or optimization for the Java VM. It stores per-class structures such as a runtime constant pool, field and method data, and the code for methods and constructors. The method area is logically part of the heap but, depending on the implementation, a Java VM may not garbage collect or compact it. Like the heap memory, the method area may be of a fixed or variable size. The memory for the method area does not need to be contiguous.\nPermanent Generation: The pool containing all the reflective data of the virtual machine itself, such as class and method objects. With Java VMs that use class data sharing, this generation is divided into read-only and read-write areas. Code Cache: The HotSpot Java VM also includes a code cache, containing memory that is used for compilation and storage of native code. And for the application, the below diagram will give you an idea about how memory is distributed.\nso now lets go back to the method findTenuredGenPool().\n1 2 3 4 5 6 7 8 9 10 11 private static MemoryPoolMXBean findTenuredGenPool() { for (MemoryPoolMXBean pool : ManagementFactory.getMemoryPoolMXBeans()) { if (pool.getName().contains(\u0026#34;Old\u0026#34;) \u0026amp;\u0026amp; pool.getType() == MemoryType.HEAP \u0026amp;\u0026amp; pool.isUsageThresholdSupported()) { return pool; } } throw new Exception(\u0026#34;Could not find tenured space\u0026#34;); } } in this method we are looking for a pool which is of type HEAP and also in its name contains the word old. This is one way we can find the Tenured gen pool.\nSo using this pools size and its threshold we can make alert service that will give us alert when memory is low. That way we can take necessary steps to prevent OOM error.\nSo now running this service\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 class Day63 { public static void main(String[] args) throws Exception { OOMAlertService.setUsageThreshold(60.0/100.0); OOMAlertService mws = new OOMAlertService(); mws.addListener((usedMemory, maxMemory, stacktrace) -\u0026gt; { System.out.println(\u0026#34;Memory usage low!!!\u0026#34;); double percentageUsed = ((double) usedMemory) / maxMemory; System.out.println(\u0026#34;percentageUsed = \u0026#34; + percentageUsed); OOMAlertService.setUsageThreshold(80.0/100.0); stacktrace.forEach((key, value) -\u0026gt; System.out.println(key + \u0026#34; \u0026#34; + Arrays.toString(value))); }); List\u0026lt;List\u0026lt;Object\u0026gt;\u0026gt; numbers = new ArrayList\u0026lt;\u0026gt;(); while (true) { numbers.add(new ArrayList\u0026lt;\u0026gt;()); } } Here I am just creating a list of object infinitely to induce OOM.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day63/","summary":"Using JMX to write an alert system for Java JVM OutOfMemory Error","title":"Day 63: Alert system for Java OutOfMemory Error "},{"content":"In part one we disscussed about JMX and its remote management layer. In this one we are going to understand its second layer which is Agent layer.\nAgent Overview A JMX agent is a standard management agent that directly controls resources. And it makes them available to remote management applications. A JMX agent is usually located on the same system as the resources that it controls, but this is not a need.\nA JMX agent is a management entity that runs in a JVM and acts as the liaison between the managed beans (MBeans) and the management application. The various components of a JMX agent are outlined in the following sections:\nMBean Server Agent Services Protocol Adaptors and Connectors MBean Server: The MBean server AKA MBean Agent is the core component of a JMX agent. It’s a registry for objects in a JMX agent which does management operations. An object that registers with the MBean server is visible to management applications. The MBean server exposes only the management interface of an MBean.\nAny resource that you want to manage from outside the agent’s JVM must be registered as an MBean with the server. The MBean server provides a standardized interface for accessing MBeans within the same JVM. Thus giving local objects all the benefits of manipulating manageable resources. MBeans can be instantiated and registered by:\nAnother MBean The agent itself A remote management application When you register an MBean, you must assign it a unique object name. A management application uses the object name to identify the object on which it is to perform a management operation. The operations available on MBeans include:\nDiscovering the management interface of MBeans Reading and writing their attribute values Performing operations defined by the MBeans Getting notifications emitted by MBeans Querying MBeans by using their object name or their attribute values Architecture What is MBean MBean Server MBean MBeans are managed beans, Java objects that represent resources to be managed. MBeans can be standard or dynamic. Standard MBeans are Java objects that conform to design patterns derived from the JavaBeans component model. Dynamic MBeans define their management interface at runtime.\nThere are two types of MBeans\nStandard MBean Dynamic MBean A standard MBean exposes the resource to be managed directly through its attributes and operations. Attributes are exposed through \u0026ldquo;getter\u0026rdquo; and \u0026ldquo;setter\u0026rdquo; methods. Operations are the other methods of the class that are available to managers. All these methods are defined statically in the MBean interface and are visible to a JMX agent through introspection. This is the most straightforward way of making a new resource manageable.\nA dynamic MBean is an MBean that defines its management interface at runtime. For example, a configuration MBean could determine the names and types of the attributes it exposes by parsing an XML file.\nA JMX agent is a management entity that runs in a JVM and acts as the liaison between the managed beans (MBeans) and the management application. The various components of a JMX agent are outlined in the following sections:\nMBean Server Agent Services Protocol Adaptors and Connectors MBean Server The MBean server is the core component of a JMX agent. It’s a registry for objects in a JMX agent that are exposed to management operations. An object that is registered with the MBean server is visible to management applications. The MBean server exposes only the management interface of an MBean, never its direct object reference.\nAny resource that you want to manage from outside the agent’s JVM must be registered as an MBean with the server. The MBean server provides a standardized interface for accessing MBeans within the same JVM, giving local objects all the benefits of manipulating manageable resources. MBeans can be instantiated and registered by:\nAnother MBean The agent itself A remote management application When you register an MBean, you must assign it a unique object name. A management application uses the object name to identify the object on which it is to perform a management operation. The operations available on MBeans include:\nDiscovering the management interface of MBeans Reading and writing their attribute values Performing operations defined by the MBeans Getting notifications emitted by MBeans Querying MBeans by using their object name or their attribute values Agent Services Agent services are objects that can perform management operations on the MBeans that registers with the MBean server. Agent services can be provided by MBeans as well, allowing them and their functionality to be controlled through the MBean server. Java Management Extensions (JMX) Specification, version 1.4 defines the following agent services:\nDynamic Class loader : Dynamic class loading through the management service fetches and instantiates new classes and native libraries. That are dynamically downloaded from the network. Monitors : Monitors the numerical or string value of MBean attributes. They can notify other objects of several types of changes. Timers: Timers provide a scheduling mechanism and can send notifications at predetermined intervals.\nRelation service: The relation service lets the MBeans communicate with each other.\nProtocol Adaptors and Connectors Protocol adaptors and connectors make agents accessible from remote management applications. They provide a view through a specific protocol of the MBeans that are instantiated and registered with the MBean server. They enable a management application outside the JVM to:\nGet or set attributes of existing MBeans Perform operations on existing MBeans Instantiate and register new MBeans Register for and receive notifications emitted by MBeans\nPlatform MBeans A platform MBean (also called an MXBean) is an MBean for monitoring and managing the Java Virtual Machine (JVM). Each MXBean encapsulates a part of JVM functionality such as the JVM\u0026rsquo;s class loading system, JIT compilation system, garbage collector, and so on. The java.lang.management package defines the platform MXBeans.\nTable 1 lists all the platform MBeans and the aspect of the VM that they manage. Each platform MXBean has a unique javax.management.ObjectName for registration in the platform MBeanServer. A JVM may have zero, one, or more than one instance of each MXBean, depending on its function, as shown in the table.\nPlatform MBeans Interface Manages Object Name Instances Per VM ClassLoadingMXBean Class loading system java.lang:type=ClassLoading One CompilationMXBean Compilation system java.lang:type=Compilation Zero or one GarbageCollectorMXBean Garbage collector java.lang:type=GarbageCollector name=collectorName One or more MemoryManagerMXBean (sub-interface of GarbageCollectorMXBean) Memory pool java.lang:type=MemoryManager name=managerName One or more MemoryPoolMXBean Memory java.lang:type=MemoryPool name=poolName One or more MemoryMXBean Memory system java.lang:type=Memory One OperatingSystemMXBean Underlying operating system java.lang:type=OperatingSystem One RuntimeMXBean Runtime system java.lang:type=Runtime One ThreadMXBean Thread system java.lang:type=Threading One Platform MBean Server The Platform MBean Server can be shared by different managed components running within the same Java Virtual Machine. You can access the Platform MBean Server with the method ManagementFactory.getPlatformMBeanServer(). The first call to this method, creates the platform MBeanServer and registers the platform MXBeans using their unique ObjectNames. Subsequently, it returns the initially created platform MBeanServer.\nMXBeans that get created and destroyed dynamically, for example, memory pools and managers, will automatically be registered and deregistered into the platform MBeanServer. If the system property javax.management.builder.initial is set, the platform MBeanServer creation will be done by the specified MBeanServerBuilder.\nUse the platform MBeanServer to register other MBeans besides the platform MXBeans. This enables all MBeans to be published through the same MBeanServer and makes network publishing and discovery easier.\nNow in this Java code we create a MBean and see it In visualVm in step 1\nlets create a SystemStatusMBean to get some system related information\n1 2 3 public interface SystemStatusMBean { Long getuptime(); } This tells the MBeansServer by using dependency injection that this object will export 1 metrics:\nuptime Let’s write the actual object that implements this behaviour:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 public class SystemStatus implements SystemStatusMBean { private Long uptime; private Thread backgroundThread; public SystemStatus() { // First we initialize all the metrics this.backgroundThread = new Thread(); this.uptime = 0; // We will use a background thread to update the metrics this.backgroundThread = new Thread(() -\u0026gt; { try { while (true) { // Every second we update the metrics uptime += 1; Thread.sleep(1000L); } } catch (Exception e) { e.printStackTrace(); } }); this.backgroundThread.setName(\u0026#34;backgroundThread\u0026#34;); this.backgroundThread.start(); } // Through this getters, defined in the interface SystemStatusMBean, // all the metrics will be automatically retrieved @Override public Long getUptime() { return uptime; } } From the previous piece of code, you can see that you are simply creating an object that implements all the metrics from the interface, and will update them every second using a background thread.\nFinally, we create an instance of the MBean object and register it in the MBeanServer:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 import javax.management.*; import java.lang.management.ManagementFactory; public class Day62 { public static void main(String[] args) { try { // Initialize the object SystemStatus systemStatus = new SystemStatus(); // Register the object in the MBeanServer MBeanServer platformMBeanServer = ManagementFactory.getPlatformMBeanServer(); ObjectName objectName = new ObjectName(\u0026#34;com.simple.app:name=SystemStatus\u0026#34;); platformMBeanServer.registerMBean(systemStatus, objectName); } catch (Exception e) { e.printStackTrace(); } } } The MBean needs a ObjectName to identify it within the MBeanServer. The name must contain a domain to avoid collisions and keys. In this example, the domain would be com.simple.app and the key would be name=SystemStatus.\nnow lets see the MBean from VisualVM After running the program it will run a thread in the background as while(true) so after selecting the running program from visualvm we will get something like this\nI have run this program in intellij Scratch so visualVM is registering it as that. Now if i click on the mbean menu i will see something like\nwhere we can see com.simple.app which we have given in the PlafromMBean registration. After expanding it we will see the name of the object which is SystemStatus\nafter opening the SystemStatus we can see the uptime that we have defined for getting the uptime of the program. This metric will be updated as per the program. So if we press refresh then we can see the updated value of the uptime metric.\nHere the uptime metric changed from 14 to 61 seconds. So we can define metrics using mbeans and get information.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day62/","summary":"Understanding JMX and its architecture","title":"Day 62: Understanding JMX and its architecture ( Agent Level )"},{"content":"What is JMX The Java Management Extensions (JMX) API is a standard for managing and monitoring applications and services.\nIt defines -\nmanagement architecture design patterns APIs services for building web-based, distributed, dynamic, and modular solutions to manage Java-enabled resources. The JMX APIs make it possible to add manageability to Java-enabled applications, from web phones to set-top boxes to network devices and servers.\nThe JVM gives you a set of MBeans through which you can access runtime data like memory consumption, GC stats and some more data. You can also invoke many operations. Your app server will also give you many MBeans which you can use to control the server and installed applications.\nJMX overview The JMX technology is native to the Java programming language.\nThe java.lang.management package provides the interface for monitoring and managing the JVM.\nThe API provides access to information such as:\nNumber of classes loaded and threads running Virtual machine uptime, system properties, and JVM input arguments Thread state, thread contention statistics, and stack trace of live threads Memory consumption Garbage collection statistics Low memory detection On-demand deadlock detection Operating system information The API includes logging monitoring and management capabilities. The java.util.logging.LoggingMXBean interface provides for management of the logging facility.\nAs a result, it offers natural, efficient, and lightweight management extensions to Java-based apps. It consists of a set of specifications and development tools for managing Java environments and developing state-of-the-art management solutions for applications and services. It provides Java developers with the means to instrumet Java code, create smart Java agents, implement distributed management middleware and managers, and integrate these solutions into existing management and monitoring systems (eg. APM)\nThe dynamics of the JMX technology architecture enables you to use it to monitor and manage resources as they are implemented and installed. It can also be used to monitor and manage the jvm\nJMX\u0026rsquo;s layerd architecture JMX technology provides a tiered architecture where managed resources and management applications can be integrated in the plug-and-play approach as shown in below image . A given resource is instrumented by one or more Java objects known as Managed Beans (or MBeans), which are registered in a core managed object server known as the MBean server. This server acts as a management agent and can run on most Java-enabled devices.\nSo in this part the topic of discussion will be the Manager Level. But before going into it let\u0026rsquo;s get a overview of all the levels\nLevel Description Instrumentation Resources, such as applications, devices, or services, are instrumented using Java objects called Managed Beans (MBeans). MBeans expose their management interfaces, composed of attributes and operations, through a JMX agent for remote management and monitoring. Agent The main component of a JMX agent is the MBean server. This is a core managed object server in which MBeans are registered. A JMX agent also includes a set of services for handling MBeans. The JMX agent directly controls resources and makes them available to remote management agents. Manager Protocol adaptors and standard connectors make a JMX agent accessible from remote management applications outside the agent’s Java Virtual Machine (JVM). Manager level: JMX API instrumentation can be accessed through existing management protocols. For example Simple Network Management Protocol (SNMP). Another way is through proprietary protocols. The MBean server relies on protocol adaptors and connectors to make a JMX agent accessible from management applications outside the agent’s JVM.\nEach adaptor provides a view through a specific protocol of all MBeans registered in the MBean server. For example, an HTML adaptor could display an MBean in a browser.\nConnectors provide a manager-side interface that handles the communication between the manager and the JMX agent. Each connector provides the same remote management interface though a different protocol. When a remote management application uses this interface, it can connect to a JMX agent transparently through the network, regardless of the protocol.\nA graphical JMX monitoring tool, jconsole, enables us to monitor the performance of a JVM and instrumented applications, providing information to help you optimize performance.\nNow we are going to explore one of the connectors which will be used by VisualVM which is different from Jconsole. JConsole uses only JMX, but VisualVM uses other monitoring technologies like Jvmstat, Attach API and SA in addition to JMX. It can merge data from all those monitoring technologies in one place and the user does not need to think which technology he should use in particular situation.\nVisualVM helps to get lot of information out about the JVM and how it\u0026rsquo;s performing. But we will only look into the jmx connection in VisualVM\nSetup VisualVM with JMX : After downloading VisualVM when starting it will look for all the local JVM instances. Remote connections need to be added by specifying host and jmx port which needs to be setup while running the application as system property.\n1 2 3 4 -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9875 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false this configuration is for non-production use case as the port is open withou any SSL security. Here the jmx port will be 9875 and then we will need to connect to this port from visualvm. This port is by default given.\nnow after connecting with jmx port in visualvm we will get to see something like this\nwhere there is tabs like monitor ,threads, sampler, profiler and many more extensions. By using these we can monitor\nCpu Usage Heap Size Threads and states of them Sample CPU and Memory usage Profiling Check Platform and Custom MBeans for JVM and applications VisualGC which helps to visualize GC here we can see the memory and cpu usage of intellij IDE.\nIn the next part will try to discuss about the Agent level of JMX\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day61/","summary":"Understanding JMX and its architecture","title":"Day 61: Understanding JMX and its architecture (Management Level) "},{"content":"Heap in JVM At first before we dump the heap we should unnderstand what is the Heap and why should we dump it. In JVM The Java heap is the area of memory used to store objects instantiated by applications running on the JVM. When the JVM is started, heap memory is created and any objects in the heap can be shared between threads as long as the application is running. The size of the heap can vary, so many users restrict the Java heap size to 2-8 GB in order to minimize garbage collection pauses.\nWhy dump Heap Memory A heap dump is a snapshot of all the objects that are in memory in the JVM at a certain moment. They are very useful to troubleshoot memory-leak problems and optimize memory usage in Java applications.\nHeap dumps are usually stored in binary format hprof files. We can open and analyze these files using tools like JVisualVM and Eclipse Memory Analyzing Tool\nTo generate a heap dump we first need to get a connection to heap and then create the heap dump file. So to that we need the help of java\u0026rsquo;s JMX framework.\nWhat is JMX Java Management Extensions (JMX) is a standard component of the Java Platform. It was first added to the J2SE 5.0 release. It is a set of specifications used for network and application management. It specifies a method for developers to integrate the applications they are working on with their network management software by assigning Java objects with management attributes.\nJMX gives developers a standard and simple way to manage resources. Including services, devices, and applications. It is dynamic, making it possible to manage and monitor resources as soon as they are created, implemented or installed.\nWhat is MBean and MBeanServer With Java Management Extensions technology, a resource is represented by Managed Beans or mBeans. These are registered on the mBean server. It is a core-managed object server that acts as an agent and can be used on a majority of devices that support Java.\nIn simpler terms, mBeans acts like Java wrappers for\nservices components devices applications in a distributed network.\nMBean server provides the actual management, as it is where you would find all the manageable resources. This server then becomes the central focus of the architectural frame. which allow the server components to plug in and find manageable objects.\nA JMX agent, would consist of the mBean server, and the services needed to handle the mBeans (you’ll also want an APM solution that includes application framework metrics like mBeans and performance counters). This means that the resources are independent and apart from the management infrastructure. while these resources are manageable no matter how the management applications are deployed.\nSo now by using these MBeans and MBeanServer\u0026rsquo;s we can get many inforamtions about the JVM and the application that its hosting.\nDoing Heap Dump Programatically At first we need to call the PlatformMBeanServer to get the platform informations of JVM and then with those information need to access HotSpotDiagnosticMXBean whic is an Management Extension Bean different from typical MBean.\nSo let us understand what is and MXBean\nMXBean MXBeans are just a special kind of MBeans. The main difference is that MXBean restrict the data types, so that they are \u0026ldquo;more compatible\u0026rdquo; with potential clients.\nAs example: a MBean can expose attributes of a data type Foo. Now the client also needs to have this type Foo to make sense of the attribute.\nThe MXBean tries to restrict the data types to those already available e.g. - java.lang.* etc.\nprocedure To extract a heap dump\nAt first need to get PlatformMXBeanServer. From that need to get the specific MXBean in this case HotSpotDiagnotsicMXBean As the procedure is not thread safe we need to sychronize it Now we need to call the dumpHeap method on HotSpotDiagnotic To call this method we need to pass file name (with .hprof extension) and boolean option to get information about live objects in the heap. It will return the file in the specified locaiton Code 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 class Day60 { private static final String HotSpotBeanName = \u0026#34;com.sun.management:type=HotSpotDiagnostic\u0026#34;; private static volatile HotSpotDiagnosticMXBean hotSpotDiagnosticMXBean; public static void main(String[] args) throws IOException { String timestamp = LocalDateTime.now().toString(); System.out.println(\u0026#34;Dumping heap\u0026#34;); generateHeapDump(\u0026#34;/home/mohibul/Documents/heapdump_\u0026#34;+timestamp+\u0026#34;.hprof\u0026#34;, true); } private static void generateHeapDump(String fileName, boolean isLive) throws IOException { if (hotSpotDiagnosticMXBean == null) { synchronized (Day60.class) { hotSpotDiagnosticMXBean = getHotSpotDiagnosticMXBean(); } } hotSpotDiagnosticMXBean.dumpHeap(fileName,isLive); } private static HotSpotDiagnosticMXBean getHotSpotDiagnosticMXBean() throws IOException { MBeanServer mBeanServer = ManagementFactory.getPlatformMBeanServer(); return ManagementFactory.newPlatformMXBeanProxy(mBeanServer, HotSpotBeanName, HotSpotDiagnosticMXBean.class); } } ","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day60/","summary":"Generating JVM heap dump using PlatformMBeanServer","title":"Day 60: Generating JVM heap dump programmatically "},{"content":"Proxy Proxy is a design pattern. We create and use proxy objects when we want to add or change some functionality of an existing class. The proxy object is used instead of the original one. Usually, the proxy objects have the same methods as the original one. In Java proxy classes usually extend the original class. The proxy has a handle to the original object and can call the method on that.\nA proxy, in its most general form, is a class functioning as an interface to something else. In layman’s term, a proxy class in java is a class that delegates responsibility “in-place of” or “on behalf of” another class. The object, a proxy imitates is called the implementation object.\nTypes of Proxy in java There are mainly 2 types of Proxy in Java\nStatic Proxy Dynamic Proxy Static Proxy Proxies that are written manually are referred to as static proxies. The following example is for a statc proxy\nAt first we can create an interface to be shared among the proxy and the real class.\n1 2 3 public interface User { String getType(); } now that we have an interface we can have its implementation of a particular user.\n1 2 3 4 5 public class FreeUser implements User { public String getType() { return \u0026#34;free\u0026#34;; } } Now we can create a Proxy interface that will extend User interface to get its behaviors which can be proxied by its implementations.\n1 2 public interface ProxyUser extends User { } So FreeUserProxy can be implemented by implementing ProxyUser\n1 2 3 4 5 6 7 8 9 public class FreeUserProxy implements ProxyUser { private FreeUser freeUser = new FreeUser(); private static final Logger log = LoggerFactory.getLogger(FreeUserProxy.class); public String getType() { log.info(\u0026#34;getType() called\u0026#34;); return freeUser.getType(); } } Now running the code from main class\n1 2 3 4 5 6 public class Day59 { public static void main(String [] args) { User user = new FreeUserProxy(); System.out.println(user.getType()); } } Usage Proxy pattern is used when we need to create a wrapper to cover the main object’s complexity from the client. Furthermore we can add additional behavior on the proxy object that can augment the proxied object.\nAdvantages A proxy can hide complex tasks such as making network communication, transaction management without changing the implementation.\nProxies can be used to insert custom behaviour/code on top of and without changing the implementation object. Sometimes the code of an external library is inaccessible to edit, custom behaviour can be inserted pre/post-execution of the method provided by such library. For example, you can write a proxy for java.net.HttpUrlConnection class to log all the external service call request without changing the implementation of HttpUrlConnection.\nOne of the other advantages of the proxy pattern is security. A remote proxy can be used to provide a proxy stub in client and call the implementation on the server.\nDisadvantages Sometimes static proxy can violate the dry principle, e.x The static proxy class defined is very specific to an implementation which means for every implementation a proxy needs to be explicitly defined and is a repeated work.\nConsider a scenario where you have to implement a proxy to count method invocation for multiple class. If you are using a static proxy you will be defining multiple proxy class with duplicate logic over and over again.\nIn the example above through proxy, we are counting method invocation using a single line. If a proxy had 100 lines of code to persist a data in the database, and a bug was found in just one line, you would have to remember to change that line in each bit of duplicated code, throughout tens, or even hundreds, of additional proxies.\nDynamic proxy Dynamic proxy is the proxy design pattern, in which the proxy object is created dynamically during runtime.\nProxy design pattern uses a proxy. which acts as a mediator between client and underlying real object. Programmer can perform access control, validation and additional action in proxy before delegating the request to real object.\nForm the disadvantages of the static proxy, if we somehow at runtime we are able to create a proxy object based on the client\u0026rsquo;s call and then perform generic action(logging action in our case) before delegating the call to the real object? Well, that is what dynamic proxies does.\nThe process in case of dynamic proxy is as following:\nclient calls some action on an object. system creates a proxy object at runtime based on client\u0026rsquo;s call. proxy object calls a generic method to perform a generic action in case of each call. after the action, proxy object delegates the call to real object. So in a nutshell, if you have some generic action to perform, use dynamic proxy, but if you want each class to be treated differenlty (in some classes perform logging, in some don\u0026rsquo;t, in some access control etc.) use simple proxy.\nNow to create a dynamic proxy in java we can use Java Reflection to create dynamic implementations of interfaces at runtime. By using the class java.lang.reflect.Proxy.\nDynamic proxies can be used for many different purposes, e.g. database connection and transaction management, dynamic mock objects for unit testing, and other AOP-like method intercepting purposes.\nWe create dynamic proxies using the Proxy.newProxyInstance() method. The newProxyInstance() methods takes 3 parameters:\nThe ClassLoader that is to load the dynamic proxy class. An array of interfaces to implement. An InvocationHandler to forward all methods calls on the proxy to. For example\n1 2 3 4 5 InvocationHandler handler = new UserInvocationHandler(); User proxy = (User) Proxy.newProxyInstance( User.class.getClassLoader(), new Class[] { User.class }, handler); After running this code the proxy variable contains a dynamic implementation of the Vehicle interface. All calls to the proxy will be forwarded to the handler implementation of the general InvocationHandler interface.\nInvocationHandler As mentioned earlier you must pass an InvocationHandler implementation to the Proxy.newProxyInstance() method. All method calls to the dynamic proxy are forwarded to this InvocationHandler implementation. Here is how the InvocationHandler interface looks:\n1 2 3 4 public interface InvocationHandler{ Object invoke(Object proxy, Method method, Object[] args) throws Throwable; } The proxy parameter passed to the invoke() method is the dynamic proxy object implementing the interface. Most often you don\u0026rsquo;t need this object.\nThe Method object passed into the invoke() method represents the method called on the interface the dynamic proxy implements. From the Method object you can obtain the method name, parameter types, return type, etc.\nThe Object[] args array contains the parameter values passed to the proxy when the method in the interface implemented was called. Note: Primitives (int, long etc) in the implemented interface are wrapped in their object counterparts (Integer, Long etc.).\nNow for example writing a GenericLogger class that will log method invocation by intercepting the method invocation.\n1 2 3 4 5 6 7 8 9 10 11 12 public class GenericLogger implements InvocationHandler { private Object target; public GenericLogger(Object target) { this.target = target; } public Object invoke(Object proxy, Method m, Object[] args) throws Throwable { System.out.println(\u0026#34;Generic Logger Entry: Invoking \u0026#34; + m.getName()); return m.invoke(target, args); } } now to invoke the proxy from main class\n1 2 3 4 5 6 7 8 9 public class Day59 { public static void main(String[] args) { User user = new FreeUser(); ClassLoader cl = User.class.getClassLoader(); User u = (User) Proxy.newProxyInstance(cl, new Class[] {User.class}, new GenericLogger(c)); u.getType(); } } Known Use Cases Dynamic proxies are known to be used for at least the following purposes:\nDatabase Connection and Transaction Management Dynamic Mock Objects for Unit Testing Adaptation of DI Container to Custom Factory Interfaces AOP-like Method Interception Advantages It can be used as decorator to provide some extra ability to proxied objects and all the other benifits of static proxy and not its disadvantages.\nDisadvantages It uses the java Reflection Api which tends to be slow in performance and object creation. Will also feel magical if not have a good understanding how dynamic proxy is working and creating proxy in the runtime\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day59/","summary":"Understanding Proxy and Dynamic Proxy in Java","title":"Day 59: Understanding Proxy and Dynamic Proxy in Java"},{"content":"When unit testing any java class which have void methods inside it and also needed to be mocked. Then we mock them in 4 ways using mockito\u0026rsquo;s mocking method\u0026rsquo;s and they are -\ndoNothing() \u0026amp; ArgumentCapture doThrow() doAnswer() doCallRealMethod() doNothing() : when using doNothing() as its name suggest it does nothing. So when verifying we need to verify if the method is called or not.\n1 2 3 4 5 6 7 void add(Number a, Number b) { if (a instanceof Integer \u0026amp;\u0026amp; b instanceof Integer) { addInt(a.intValue(), b.intValue()); } else if (a instanceof Double \u0026amp;\u0026amp; b instanceof Double) { addDouble(a.doubleValue(), b.doubleValue()); } } if this add() method is tested with doNothing() it will just invoke it and then verify whether it was invoked or not. So the test would be\n1 2 3 4 5 6 7 @Test public void whenAddCalledVerified() { Scratch scratch = mock(Scratch.class); doNothing().when(scratch).add(anyInt(), anyInt()); verify(scratch, times(1)).add(1, 1); } doNothing() + ArgumentCaptor : So before going into how this helps mocking void method. Let us learn about what is ArgumentCaptor\nArugmentCaptor ArgumentCaptor allows us to capture an argument passed to a method to inspect it. This is useful when we can\u0026rsquo;t access the argument outside of the method we\u0026rsquo;d like to test.\nTo see it in action first we need to setup a method and test.\n1 2 3 4 5 6 7 8 9 10 11 public class BookService { private final BookMeta bookMeta; @Override public void printBook(long id, Book book) { BookMeta meta = getBookMetaInfoById(id); meta.addBookMetaInfo(book); System.out.println(book.toString()); } So now to test this printBook method we need to setup a test\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 @RunWith(MockitoJUnitRunner.class) public class BookTest { String bookTitle = \u0026#34;Kill It With Fire\u0026#34; @Mock BookMeta bookMeta; @InjectMocks BookService bookService; @Captor ArgumentCaptor\u0026lt;Book\u0026gt; bookCap; bookService.printBook(1, new Book(bookTitle)); Mockito.verify(bookMeta).addBookMetaInfo(bookCap.capture()); Book capturedBook = bookCap.getValue(); assertEquals(capturedBook.getTitle(), bookTitle); } in this code we can see there is an implementation of a BookService. In that a method is implemented named findBookById which uses another 2 services named BookEntityService and BookMetaEntityService each provides some book and some related information.\nwe define an ArgumentCaptor\u0026lt;Book\u0026gt; with annotation @Captor. then we capture it while verifying the call addBookMetaInfo call and then verify the value with assertion.\nThis use case is well suited when the arguments is manipulated in the void method we are mocking.\ndoThrow(): This method can help to verify if a method throws any exception and based on that we can test some execution path.\nIn the add() method we can check for addition overflow and throw a exception so that we can verify the execution path.\n1 2 3 4 5 6 7 @Test(expected = Exception.class) public void givenNull_addThrowsException() { Scratch scratch = mock(Scratch.class); doThrow().when(scratch).add(any(), null); scratch.add(1, null); } doAnswer() Answer is used when you need to do additional actions when a mocked method is invoked, e.g. when you need to compute the return value based on the parameters of this method call. So Use doAnswer() when we want to stub a void method with generic Answer. Answer specifies an action that is executed and a return value that is returned when you interact with the mock.\nso we can make answer to add() method and to verfiy execution.\n1 2 3 4 5 6 7 8 9 10 11 12 13 @Test public void whenAddCalledAnswered() { Scratch scratch = mock(Scratch.class); doAnswer(invocation -\u0026gt; { Object arg0 = invocation.getArgument(0); Object arg1 = invocation.getArgument(1); assertEquals(1, arg0); assertEquals(1, arg1); return null; }).when(scratch).add(anyInt(), anyInt()); scratch.add(1,1); } Also another common usage of Answer is to stub asynchronous methods that have callbacks. For example, we have mocked the interface below:\n1 2 3 public interface BookService { void get(Callback callback); } Here you’ll find that when-thenReturn is not that helpful anymore. Answer is the replacement. For example, we can emulate a success by calling the onSuccess function of the callback.\n1 2 3 4 5 6 7 8 9 10 @Test public void testOnSuccess() { BookService bookService = mock(BookService.class); doAnswer(invocation -\u0026gt; { Callback callback = (Callback) invocation.getArguments(0); callback.onSuccess(data); return null; }).when(bookService).get(any(Callback.class)); } doCallRealMethod() By calling this method we can achive partial mocking of method. When we Mock an object and then make a reference to any method using mocked object reference. java never makes a call to that method and looks for mocked value to be returned or null if none specified. But If we want that to be overridden and want java to make actual method call upon using mocked object reference, this method can be used.\n1 2 3 4 5 6 7 8 @Test public void whenAddCalledRealMethodCalled() { Scratch scratch = mock(Scratch.class); doCallRealMethod().when(scratch).add(anyInt(), anyInt()); scratch.add(1,1); verify(scratch, times(1)).add(1, 1); } Although I found only these way\u0026rsquo;s of mocking void methods in junit and mockito. There might be many other ways we can achive mocking the void method calls.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day58/","summary":"Mocking void methods using ArgumentCapture and Answering","title":"Day 58: Mocking void methods using ArgumentCapture and Answering"},{"content":"To generate random numbers in java there are 4 given ways by the api. In java 17 new generic api and some other api\u0026rsquo;s was introduced.\nRandom Class ThreadLocalRandom Class SecureRandom Class SplitabbleRandom Class Fig: Java 17 new RandomGenerator Interface.\nRandom: The Random Class of the java.util package is used for generating a stream of pseudorandom numbers. It uses a 48-bit seed, and using a Linear Congruential Formula.\nIf we create two instances of the Random class with the same seed value and call the same sequence of methods for each . They both will return identical sequences of numbers. This property is enforced by specific algorithms defined for this class. The algorithms use a protected utility method which upon invocation can give up to 32 pseudorandomly generated bits.\nInstances of Random are thread–safe. Although, if the same instances are used across threads they may suffer from contention and result in poor performance. The Instances of Random are not cryptographically safe. So should not be used for security-sensitive applications.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 public class Day57 { public static void main(String[] args) { long seed = 5; Random rng1 = new Random(seed) System.out.println(rng.nextInt()); Random rng2 = new Random(seed) System.out.println(rng.nextInt()); new Random(seed).ints(5, 10, 100) .forEach(System.out::println); } } ThreadLocalRandom: ThreadLocalRandom is a combination of the ThreadLocal and Random classes. Its isolated to the current thread. So it achieves better performance in a multi-threaded environment. It avoids any concurrent access to instances of Random.\nThe random number obtained by one thread is not affected by the other thread. Whereas java.util.Random provides random numbers globally.\nAlso, unlike Random, ThreadLocalRandom doesn\u0026rsquo;t support setting the seed explicitly. Instead, it overrides the setSeed(long seed) method. Its inherited from Random and always throw an UnsupportedOperationException if called.\nSo now we will take a detour and try to understand what is Thread contention which is the reason behind Random classes result in poor performance while being in multi-threaded env\nThread Contention: when two threads try to access either the same resource or related resources. At least one of the contending threads runs slower than it would if the other thread(s) were not running. This situation is called thread contention.\nThe most obvious example of contention is on a lock. If thread A has a lock and thread B wants to acquire that same lock, thread B will have to wait until thread A releases the lock.\nFor example, consider a thread that acquires a lock, modifies an object, then releases the lock and does some other things. If two threads are doing this, even if they never fight for the lock, the threads may run much slower than they would if only one thread was running.\nWhy? Say each thread is running on its own core on a modern x86 CPU and the cores don\u0026rsquo;t share an L2 cache. With just one thread, the object may remain in the L2 cache most of the time. With both threads running, each time one thread modifies the object, the other thread will find the data is not in its L2 cache because the other CPU invalidated the cache line.\nSo far, we can see that the Random class performs poor in highly concurrent environments. To better understand this, let\u0026rsquo;s see how one of its primary operations, next(int), is implemented\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 /** * Generates the next pseudorandom number. Subclasses should * override this, as this is used by all other methods. * * \u0026lt;p\u0026gt;The general contract of {@code next} is that it returns an * {@code int} value and if the argument {@code bits} is between * {@code 1} and {@code 32} (inclusive), then that many low-order * bits of the returned value will be (approximately) independently * chosen bit values, each of which is (approximately) equally * likely to be {@code 0} or {@code 1}. The method {@code next} is * implemented by class {@code Random} by atomically updating the seed to * \u0026lt;pre\u0026gt;{@code (seed * 0x5DEECE66DL + 0xBL) \u0026amp; ((1L \u0026lt;\u0026lt; 48) - 1)}\u0026lt;/pre\u0026gt; * and returning * \u0026lt;pre\u0026gt;{@code (int)(seed \u0026gt;\u0026gt;\u0026gt; (48 - bits))}.\u0026lt;/pre\u0026gt; * * This is a linear congruential pseudorandom number generator, as * defined by D. H. Lehmer and described by Donald E. Knuth in * \u0026lt;i\u0026gt;The Art of Computer Programming,\u0026lt;/i\u0026gt; Volume 2: * \u0026lt;i\u0026gt;Seminumerical Algorithms\u0026lt;/i\u0026gt;, section 3.2.1. * * @param bits random bits * @return the next pseudorandom value from this random number * generator\u0026#39;s sequence * @since 1.1 */ protected int next(int bits) { long oldseed, nextseed; AtomicLong seed = this.seed; do { oldseed = seed.get(); nextseed = (oldseed * multiplier + addend) \u0026amp; mask; } while (!seed.compareAndSet(oldseed, nextseed)); return (int)(nextseed \u0026gt;\u0026gt;\u0026gt; (48 - bits)); } This portion of code is copied from openJDK and here we can see it implements Linear Congruential Generator algo to generate pseudo random numbers.It\u0026rsquo;s obvious that all threads are sharing the same seed instance variable.\nTo generate the next random set of bits, it first tries to change the shared seed value atomically via compareAndSet or CAS for short.\nWhen multiple threads attempt to update the seed concurrently using CAS, one thread wins and updates the seed, and the rest lose. Losing threads will try the same process over and over again until they get a chance to update the value and ultimately generate the random number.\nThis algorithm is lock-free , and different threads can progress concurrently. However, when the contention is high, the number of CAS failures and retries will hurt the overall performance significantly.\nOn the other hand, the ThreadLocalRandom completely removes this contention, as each thread has its own instance of Random and, consequently, its own confined seed.\ncurrent() in ThreadLocalRandom This method returns ThreadLocalRandom instance for the current thread. ThreadLocalRandom can be initialized as below.\n1 ThreadLocalRandom random = ThreadLocalRandom.current(); nextInt(int least,int bound) returns the next pseudo number . We can pass the least limit and max limit. There are more method like nextDouble, nextLong. So finally we can get random number as below\n1 int i = ThreadLocalRandom.current().nextInt(1, 10); now writing a simple implementation of ThreadLocalRandom.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 public class Day57 { public static void main(String[] args) { ForkJoinPool pool = new ForkJoinPool(); TestTask task1 = new TestTask(\u0026#34;Task one\u0026#34;); TestTask task2 = new TestTask(\u0026#34;Task two\u0026#34;); pool.invoke(task1); pool.invoke(task2); } }\tclass TestTask extends ForkJoinTask\u0026lt;String\u0026gt; { private String str = \u0026#34;\u0026#34;; public TestTask(String str){ this.str = str; } private static final long serialVersionUID = 1L; @Override protected boolean exec() { int i = ThreadLocalRandom.current().nextInt(1, 10);\tSystem.out.println(\u0026#34;ThreadLocalRandom for \u0026#34;+str+\u0026#34;:\u0026#34;+i); return true; } @Override public String getRawResult() { return \u0026#34;\u0026#34;; } @Override protected void setRawResult(String value) { } } In the example there is a ForkJoinTask implementation. Also inside exec() method of ForkJoinTask, we obtained the random number by ThreadLocalRandom. We have run two ForkJoinTask to test the random number generation. Run the example many time and every time you will get random numbers. Sample output is as below.\nSecureRandom Standard JDK implementations of java.util.Random use a Linear Congruential Generator (LCG) algorithm for providing random numbers. The problem with this algorithm is that it’s not cryptographically strong. In other words, the generated values are much more predictable, therefore attackers could use it to compromise our system.\nThe java.security.SecureRandom class does not actually implement a pseudorandom number generator (PRNG) itself. It uses PRNG implementations in other classes to generate random numbers. So the randomness of the random numbers and security and performance of SecureRandom depends on the algorithm chosen. If you want cryptographically strong randomness, then you need a strong entropy source. Entropy here refers to the randomness collected by an operating system or application. The entropy source is one which collects random data and supplies to destination.\nThe file which controls the configuration of the SecureRandom API is located at: $JAVA_HOME/lib/security/java.security\nWe can select the source of seed data for SecureRandom by using the entropy gathering device specified in securerandom.source property in java.security file.\nE.g.\nsecurerandom.source=file:/dev/random Way of initializing a SecureRandom and get a cryptographycally strong secure random numbers.\n1 2 3 SecureRandom random = new SecureRandom(); byte[] values = new byte[20]; random.nextBytes(values); another way of creating instance will be\n1 2 3 SecureRandom random = SecureRandom.getInstanceStrong(); byte[] values = new byte[20]; random.nextBytes(values); this getInstanceStrong() is a blocking call.\nDifference between them : the first instance uses /dev/urandom. The second instance uses /dev/random. /dev/random blocks the thread if there isn\u0026rsquo;t enough randomness available. Whereas /dev/urandom will never block.\nBelieve it or not, there is no advantage in using /dev/random over /dev/urandom. They use the same pool of randomness under the hood. They are equally secure. If you want to safely generate random numbers, you should use /dev/urandom.\nThe only time you would want to call /dev/random is when the machine is first booting and entropy has not yet accumulated. Most systems will save off entropy before shutting down. so that some is available when booting. so this is not an issue if you run directly on hardware.\nBut, it might be an issue if you don\u0026rsquo;t run directly on hardware. If you are using a container based solution like Docker or CoreOS. You might start off from an initial image, and so may not be able to save state between reboots. And in a multi-tenant container solution, there is only one shared /dev/random which may block.But the work around in these cases is to seed /dev/random with a userspace solution, either using an entropy server for pollinate, or a CPU time stamp counter for haveged. Either way, by the time the JVM starts, the system\u0026rsquo;s entropy pool should already be up to the job.\n1 2 3 // NativePRNG is an algorithm name to choose from SecureRandom secureRandom = SecureRandom.getInstance(\u0026#34;NativePRNG\u0026#34;); IntStream secureRandomInts = secureRandom.getInts(streamSize); SplittableRandom SplittableRandom is a really interesting class, that was added in the scope of Java 8.\nThe differences with a Random class are:\nIt passes a Diehard test, that means it can generate more quality pseudorandom numbers. it’s not thread-safe, but faster than Random and method split creates a new SplittableRandom instance that shares no mutable state with the current instance. It’s useful for parallel streams. it doesn’t extend a Random class like SecureRandom and ThreadLocalRandom. Let’s take a look at code examples.\nThe most common task is to generate a random number in the range:\n1 new SplittableRandom().nextInt(1, 5) Generate an array of random ints in the range:\n1 new SplittableRandom().ints(5, 1, 10).toArray() As you see this class has stream-friendly API. Since Java 10 was added SplittableRandom.nextBytes, so you can use it as well. Java 17 added RandomGenerator interface and SplittableRandom now extends that as we can see from the Figure.\nWill post another blog post for SplittableRandom and how its used in monte carlo simulation\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day57/","summary":"Understanding details and compare use cases  of Random, ThreadLocalRandom and SecureRandom","title":"Day 57: All about Random in Java ( Random, SecureRandom,ThreadLocalRandom and SplittableRandom)"},{"content":"Recently from one of my senior colleagues LinkedIn post I learned about zero sized array in java. This raised some curiosity. So in this post I am going to share about the learning and findings i have found about zero length array in java.\nZero Sized Array: Not only java but also many other languages have zero sized array. In C its call variable length array as the array can have initialization using zero length. In the struct it can have the size initialized.\nA zero length array is simply an array with nothing in it. It is an advantage to have such an array, especially in Java. So you can return a valid array and guarantee that your length check never fails.\nIn Java, an array even of primitive types (i.e. int[]) is still an object. It has a length property. An array in Java can be null and it is easier to return null. But to have a method return a null and then checking it from the method caller is cumbersome.\nThus a zero length array guarantees non-null returns. Which will also insure the call to array.length doesnt creates null pointer exception.\nUse of Zero Sized Array: There are many uses of zero length array. Few of them are listed below\nString.getBytes() return Calling a method of varargs parameter Null object pattern ( Sending a empty object rather than null) Empty Array from an empty list String.getBytes() return: When calling getBytes() which will return a byte[] on empty string \u0026quot;\u0026quot; what should be it\u0026rsquo;s output? It will return a zero sized byte array.\n1 System.out.println((\u0026#34;\u0026#34;.getBytes(StandardCharsets.UTF_8)).length); Calling a method of vargargs parameter: When calling a method who have a varargs parameter if we dont provide the varargs parameter with any value what will happen? It will still work. In this case the compiler is implicitly passing a zero sized array. For example the method is\n1 2 3 4 public static void printer(String s1, String ... s2) { System.out.println(s1); Arrays.stream(s2).forEach(System.out::println); } and when calling the caller is passing a single argument.\n1 2 printer(\u0026#34;bar\u0026#34;); printer(\u0026#34;foo\u0026#34;, new String[0]); // here passing the new String[0] will show a redundant warning Null Object Pattern: Instead of using a null reference to convey absence of an object (for instance, a non-existent file), one uses an object which implements the expected interface, but whose method body is empty. The advantage of this approach over a working default implementation is that a null object is very predictable and has no side effects: it does nothing.\nFor example, a function may retrieve a list of files in a folder and perform some action on each. In the case of an empty folder, one response may be to throw an exception or return a null reference rather than a list. Thus, the code which expects a list must verify that it in fact has one before continuing, which can complicate the design.\n1 2 3 4 public static File[] getFileNames(String path) { File folder = Paths.get(path).toFile(); return folder.listFiles() == null ? new File[0] : folder.listFiles(); } when executed from main\n1 2 3 //Replacement of nulls File[] files = getFileNames(\u0026#34;/home/mohibulhassan/Downloads\u0026#34;); System.out.println(files.length); Empty array from empty list: Converting an empty array from empty list one can do it using List.toArray() which will take the element of the list and create an array with those element. But if the list is empty then? By passing an zero sized array with type will convert it to a zero sized array or empty array of that type.\n1 2 List\u0026lt;String\u0026gt; list = new ArrayList\u0026lt;\u0026gt;(); String [] array = list.toArray(new String[0]); //passing type information without it needed to cast ","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day56/","summary":"What is zero length Array and why its in java and how its used","title":"Day 56: Zero length Array and Its use "},{"content":"TLDR (Too Long Didn\u0026rsquo;t Read) Blue-green deployments is a process to minimizing risk and downtime of software releases. DNS based enviorment switching is one of the way. There are many variations of this process but it is important to keep the process simple.\nBlue-green deployment First we need to understand what blue-green means. Blue-green deployment is the process of creating more than one production environment so that you can set up a new version before cutting over to it.\nBlue/green deployments provide releases with near zero-downtime and rollback capabilities. Main idea behind blue/green deployment is to move traffic between two identical env. Which are running different versions of the application.\nThe blue environment represents the current application version serving production traffic. In parallel, the green environment is running a different version of the application. After the green env is ready and tested, production traffic is then redirected from blue to green. If any problems are found, you can roll back by reverting traffic back to the blue environment.\nPro\u0026rsquo;s and Con\u0026rsquo;s of Blue/Green Deployment: In place application upgradation: Traditional deployments which upgrades the application in place makes it hard to validate your new application version in a production deployment while continuing to run the earlier version of the application.\nIsolated Environments: Blue/green deployments provide a level of isolation between your blue and green application environments. This helps ensure spinning up a parallel green environment which does not affect resources underpinning your blue environment. This isolation reduces your deployment risk.\nCanary Testing and release: After you deploy the green environment, you have the opportunity to validate it. You might do that with test traffic before sending production traffic to the green environment. Also you can use a very small fraction of production traffic, to better reflect real user traffic. This is called canary analysis or canary testing. If you discover the green environment is not operating as expected, there is no impact on the blue environment. You can route traffic back to it, minimizing impaired operation or downtime and limiting the blast radius of impact.\nRollback capabilities: This ability to simply roll traffic back to the existing production environment is a key benefit of blue/green deployments. You can roll back to the blue environment at any time during the deployment process. Impaired operation or downtime is minimized because impact is limited to the window of time between green environment issue detection and shift of traffic back to the blue environment. Additionally, impact is limited to the subset of traffic going to the green environment, not all traffic. If the blast radius of deployment errors is reduced, so is the deployment risk.\nWhat is DNS The Domain Name System (DNS) is the phonebook of the Internet. DNS translates human readable domain names (for example, www.amazon.com) to machine readable IP addresses (for example, 192.0.2.44) .\nFig: DNS Query Explained\nAll computers on the Internet, from your smart phone or laptop to the servers that serve content for massive retail websites find and communicate with one another by using numbers. These numbers are known as IP addresses. When you open a web browser and go to a website, you don\u0026rsquo;t have to remember and enter a long number. Instead, you can enter a domain name like example.com and still end up in the right place.\nWhat is CNAME and A record So to understand what is CNAME and A record we first need to understand what is a DNS record. DNS records (aka zone files) are instructions that live in authoritative DNS servers . Authoritative servers provide information about a domain. Including what IP address associates with that domain and how to handle requests for that domain. These records consist of a series of text files written in what is known as DNS syntax. DNS syntax is a string of characters used as commands that tell the DNS server what to do. All DNS records also have a ‘TTL’, which stands for time-to-live. This indicates how often a DNS server will refresh that record.\nTwo of the most common DNS records are CNAME and A record.\nA : The A stands for \u0026ldquo;address\u0026rdquo; and this is the most fundamental type of DNS record: it indicates the IP address of a given domain.\nA records only hold IPv4 addresses. If a website has an IPv6 address, it will instead use an \u0026ldquo;AAAA\u0026rdquo; record.\nHere is an example of an A record:\nexample.com record type value TTL @ A 192.0.2.1 3600 The \u0026ldquo;@\u0026rdquo; symbol in this example indicates that this is a record for the root domain, and the \u0026ldquo;14400\u0026rdquo; value is the TTL (time to live), listed in seconds. The default TTL for A records is 14,400 seconds. This means that if an A record gets updated, it takes 240 minutes (14,400 seconds) to take effect.\nThe vast majority of websites only have one A record, but it is possible to have several. Some higher profile websites will have several different A records as part of a technique called round robin load balancing. Which can distribute request traffic to one of several IP addresses, each hosting identical content.\nCNAME: The canonical name (CNAME) record is used in lieu of an A record, when a domain or subdomain is an alias of another domain. All CNAME records must point to a domain, never to an IP address. Imagine a game where each level points to another level of game. So final level points to the game boss. A domain with a CNAME record is like a level that can point you to another level (another domain with a CNAME record) or to the boss (a domain with an A record).\nFor example, suppose blog.example.com has a CNAME record with a value of ‘example.com’ (without the ‘blog’). This means when a DNS server hits the DNS records for blog.example.com, it actually triggers another DNS lookup to example.com. It returns example.com’s IP address via its A record. In this case we would say that example.com is the canonical name (or true name) of blog.example.com.\nSometimes, when sites have subdomains such as blog.example.com or shop.example.com, those subdomains will have CNAME records. Those CNAME point to a root domain (example.com). This way if the IP address of the host changes, only the DNS A record for the root domain needs to be updated. So all the CNAME records will follow along with whatever changes are made to the root.\ngreen.example.com record type value TTL CNAME example.com 3600 Deployment structure For test purpose we are going to do blue green deployment using docker containers. Here we will create 3 containers.\nGUI DNS server application Blue deployment Green Deployment GUI DNS application will help us to write DNS query. But we can easily use a cmd application which can be automated through docker compose for easier blue-green switching.\nSo the total Blue Green deployment process will be something like this\nDeploy a version of our application in a container (Blue) using A Record Create a CNAME record for our application in this case Blue container Deploy the new version of our application in a container (Green) using A record Do testing and regression on green deployment Point our CNAME record to new A (green container) record Destroy the blue container Deploy a version of our application in a container (Blue) using A Record In order to make use of blue-green deployment via DNS we first need to deploy a running version of our software. We can deploy a single container or many and create a DNS Record pointing their IP addresses. This will let us access our service without needing to know the container IP addresses.\nCreate a CNAME record for our application in this case Blue container The next DNS record we need to create is a CNAME. A CNAME is an alias from one hostname to another. This means it can be used for redirects or to provide many names for a single service. We should provision a CNAME that our customers use for accessing our service. So that can we have control of the routing.\nHere is an example of what the initial CNAME and A record might look like:\nCNAME A app.mohibul.com blue-app.mohibul.com Where blue-webapp.mohibulsblog.com is an A record of something like\nblue-app.mohibul.com record type value TTL @ A 192.0.2.1 3600 Deploy the new version of our application in a container (Green) using A record After our system is running and users are using the CNAME for accessing our service. we can deploy a new version of the software. It can be deployed right next to our current version. once deployed we should be able to access it via its own A record.\nIn this phase the green deployment is completed and we can proceed to change the dns records to do blue-green deployment. In this phase we assign the new container\u0026rsquo;s ip an A record.\ngreen-app.mohibul.com record type value TTL @ A 192.0.2.5 3600 Do testing and regression on green deployment After deployment we can do integration, functional and regression testing on the application to gain confidence whethere its working properly. If we run test on production data then we need to be very careful. Otherwise if there is dummy data to run those test and see functionaly then its ok to do.\nPoint our CNAME record to new A (green container) record To change and point users to the new application version. we need to update our CNAME record to point at the new A record. This should be quick but depending on the time to live (TTL) of the DNS record may take some time to propagate. since DNS is cached in multiple places based on the TTL provided.\nCNAME A app.mohibul.com green-app.mohibul.com Destroy the blue container After successful deployment and switching we can safely destroy the previous environment by deleting it or uninstalling any software that is left running. This can happen immediately or after some time but should be done as needed based on specific requirements.\nIn my next blog post will show a demo by running these containers and switching\n","permalink":"https://mohibul.pages.dev/posts/tutorial/devops/blue-green-release-dns/","summary":"Blue-Green Deployment using DNS to reduce downtime of production application","title":"Blue-Green Deployment using DNS"},{"content":"Static initializer Block In Java, a static initializer block executes code before the object instance initialization. A static block is a block of code with a static keyword\nstatic {\r// definition of the static block\r}\rStatic initializer block or static initialization block, or static clause are some other names for the static block. Static block code executes only once during the class loading. The static blocks always execute first before the _main() _ in Java because the compiler stores them in memory at the time of class loading and before the object creation.\nA class can have multiple static blocks, and they will execute in the same order as they appear in the class:\npublic class Day55 {\rstatic {\rSystem.out.println(\u0026quot;block 1\u0026quot;);\r}\rstatic {\rSystem.out.println(\u0026quot;block 2\u0026quot;);\r}\rpublic static void main(String[] args) {\rSystem.out.println(\u0026quot;Main Method\u0026quot;);\r}\r}\rInstance Initializer Block As the name suggests, the purpose of the instance initializer block is to initialize the instance data members.\nThe instance initializer block looks just like the static initializer block, but without the static keyword:\n{\r// definition of the Instance initialization block\r}\rStatic initializer blocks always execute before the instance initialization blocks because static blocks run at the time of class loading. However, the instance block runs at the time of instance creation. The Java compiler copies initializer blocks into every constructor. Therefore, multiple constructors can use this approach to share a block of code:\npublic class Day55 {\r{\rSystem.out.println(\u0026quot;Instance initializer block 1\u0026quot;);\r}\r{\rSystem.out.println(\u0026quot;Instance initializer block 2\u0026quot;);\r}\rpublic Day55() {\rSystem.out.println(\u0026quot;Class constructor\u0026quot;);\r}\rpublic static void main(String[] args) {\rDay55 iib = new Day55();\rSystem.out.println(\u0026quot;Main Method\u0026quot;);\r}\r}\rSo now using both to create a DateTimeFormatter for DateTimeFormatterBuilder() to get a customized DateTimeFormatter intance and initialize it in static block and then use it to calculate today\u0026rsquo;s date in the instance initializer block and printing todays date just by creating an instance of the class.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 class Day56 { private final static DateTimeFormatter DATE_TIME_FORMATTER; private static Day55 instance; public String today; // static initializer static { DATE_TIME_FORMATTER = new DateTimeFormatterBuilder() .appendLiteral(\u0026#34;Day value -\u0026gt; \u0026#34;) .appendValue(ChronoField.DAY_OF_MONTH) .appendLiteral(\u0026#34; Month value -\u0026gt; \u0026#34;) .appendValue(ChronoField.MONTH_OF_YEAR) .appendLiteral(\u0026#34; Year value -\u0026gt; \u0026#34;) .appendValue(ChronoField.YEAR) .toFormatter(); instance = new Day55(); } // Instance Initializer { today = DATE_TIME_FORMATTER.format(LocalDateTime.now()); } public static Day55 getInstance() { return instance; } } now the main method\n1 2 3 4 public static void main(String[] args) { Day55 day55FromStaticInitialization = Day55.getInstance(); System.out.println(day55FromStaticInitialization.today); } By initializing the instance as static and then making the class as Singleton to get already initialized instance when classloader loaded the classes\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day55/","summary":"Understanding java classe\u0026rsquo;s static and instance initializer","title":"Day 55: Understanding static and instance initializer in java classes and"},{"content":"While commiting in git we sometimes face some issues after commiting or pushing commits to remote repository. Some of the common scenairos are like\nCommit needs to be reverted before push Commit needs to be reverted after push Switch branch and save current uncommited work Splitting a large commit into smaller understandable commit Recovering lost file Commit in the wrong branch In these above mentioned case we can do something to fix.\nCommit needs to be reverted before push To only undo the last commit command but keep the work git reset --soft HEAD~1 Or, revert your work and undo the last commit as well git reset --hard HEAD~1 Commit needs to be reverted after push To undo the last commit which is already pushed on the remote branch git revert HEAD~1 To undo any commit pass the commit hash id git revert \u0026lt;commit-hash-id\u0026gt; Switch branch and save current uncommited work To only stash a selected file git stash push -m \u0026#34;stash-message\u0026#34; \u0026lt;file-path\u0026gt; - \u0026#34;-m stash-message\u0026#34; here the message is optional in the above command, it stands for message and adds more description to your stash\nSplitting a large commit into smaller understandable commit If by mistake you added all files in a single commit and want to split into multiple commits then use: git rebase HEAD~ Now, as usual, choose each file and commit them individually Recover lost file To recover any accidentally deleted files git restore -- \u0026lt;file-path\u0026gt; This will also get you those files which are even removed from recycle bin Commit in the wrong branch Suppose you committed last two commits in \u0026ldquo;master\u0026rdquo; instead of \u0026ldquo;feature\u0026rdquo; branch Perform the following series of commands (currently you\u0026rsquo;re on master) git stash\rgit checkout feature\rgit stash pop ","permalink":"https://mohibul.pages.dev/posts/tutorial/git/git-troubleshooting-with-helpful-commands/","summary":"Git troubleshooting with helpful commands to revert, stash, split, recover commits","title":"Git troubleshooting : 6 helpful command with explanations"},{"content":"Processors execute program instructions. So, they need to retrieve both the program instructions and required data from RAM.As CPU\u0026rsquo;s are capable of carrying out a significant number of instructions per second (IOPS), fetching from RAM isn\u0026rsquo;t that ideal for them. As its an expensive call. So to mitigate this issue there are some techniques and caching is one of them.\nHere in the below diagram we can see that each core of a processor have two level cache (L1, L2) and in L1 there is both Data and Instruction cache. Where the data and instructions are copied for processing. As different cores execute more instructions and manipulate more data, they fill up their caches with more relevant data and instructions.\nSource: Baeldung\nSimply it would be best to think about what would happen if a thread changes a cached value and at the same time other thread trying to read it\nWhat is Volatile First for understanding volatile lets understand what is non-volatile varialbe. Non volatile variables have no particular guarantee when JVM will read data from the main memory into cpu caches or when will cpu caches writes those data in main memory. This particular issue can cause lots of issues.\nSimplest example would be something like incrementing a number.\n1 2 3 4 5 public class SharedClass { public long mutableNumber = 999990; } Thread 1 increments the mutableNumber variable, but both Thread 1 and Thread 2 may read the mutableNumber variable.\nIn this case mutableNumber variable is not declared volatile so there is no guarantee about when the value of the counter variable is written from the CPU cache back to main memory. This means, that the mutableNumber variable value in the CPU cache may not be the same as in main memory.\nIts not guaranteed when cpu cache will write to memory or vice versa no thread is sure about the state of mutableNumber. Thus having a visibility issue. This is called Visibility Problem\nThe Java volatile keyword is intended to address variable visibility problems. By declaring the mutableNumber variable volatile, all writes to the mutableNumber variable will be written back to main memory immediately. Also, all reads of the mutableNumber variable will be read directly from main memory.\n1 2 3 4 public class SharedClass { public volatile long mutableNumber = 999990; } Declaring a variable volatile thus guarantees the visibility for other threads of writes to that variable.\nIn the scenario given above, where Thread 1 modifies the value, and another Thread 2 reads the value (but never modifies it), declaring the mutableNumber variable volatile is enough to guarantee visibility for Thread 2 of writes to the variable.\nWhere Volatile is Not Enough: If, however, both Thread 1 and Thread 2 were incrementing the variable, then declaring the variable volatile would not have been enough.\nIn a recent comment of mine on this question made in linkedin answered the question and now will try to expand on that. In the comment i have mentioned increment operation is not thread safe. As a simple mutableNumber++ is broken into 3 steps.\nReading the current value from memory Update the value by adding 1 with it Write the value in memory So while doing these operations threads context can switch and can cause issues for the thread who is trying to read the value while it wasnt fully committed yet in the memory by another thread. In this case the operations needs to be Atomic or the incrementing operation be behind a lock.\nAtomicity Atom comes from greek atomos = uncuttable, and has been used in the sense indivisible smallest unit. In concurrent programming, it means that there will be no context switch during it - nothing can affect the execution of atomic command.\nOn a single-processor machine, that\u0026rsquo;s a stronger guarantee than you need. On a multi-processor machine, it isn\u0026rsquo;t strong enough. What atomic really means is, no other thread will be able to see the operation in a partially-completed state\nNow to achieve the solution of the above mentione problem we can you solutions\nUsing synchronize Using AtomicLong and convert the double value to long and viceversa Using AtomicFieldUpdater (Field Updater classes can be used to perform atomic operation on a selected volatile field of a selected class.\nAtomicReferenceFieldUpdater, AtomicIntegerFieldUpdater, and AtomicLongFieldUpdater are reflection-based utilities that provide access to the associated field types) ","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day54/","summary":"Understanding volatile keyword\u0026rsquo;s use in incrementing a double variable and see how it behaves in multi threaded env","title":"Day 54: Understanding volatile keyword's use for Multi Threaded program"},{"content":"Java HttpClient can send asynchronous request using HttpClient.sendAsync() which returns a CompletableFuture\u0026lt;HttpResponse\u0026lt;T\u0026gt;\u0026gt;. By collecting all the request and then executing them with with a stream to get list of HttpResponse\u0026lt;T\u0026gt; and partition the result with status code.\nCompletableFuture: The Future interface was added in Java 5 to serve as a result of an asynchronous computation, but it did not have any methods to combine computations or handle possible errors.\nJava 8 introduced the CompletableFuture class. Along with the Future interface, it also implemented the CompletionStage interface. This interface defines the contract for an asynchronous computation step that we can combine with other steps.\nCompletableFuture is used for composing, combining, and executing asynchronous computation steps and handling errors.\nHttpClient.sendAsync(): Sends the given request asynchronously using this client with the given response body handler. This returns a CompletableFuture of HttpResponse.\nNow by sending http request from http client asynchronously we get all the CompletableFuture\u0026rsquo;s in a list and then process the list. Partitioning the list by HTTP code 200. That means if the GET request completes successfully the returned Map\u0026rsquo;s map.get(true) will list all the successfully executed HttpResponse, and map.get(false) will return the failed get request which didn\u0026rsquo;t returend HTTP 200.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 static Map\u0026lt;Boolean,List\u0026lt;HttpResponse\u0026lt;String\u0026gt;\u0026gt;\u0026gt; httpRequestDispatcher(List\u0026lt;HttpRequest\u0026gt; requests) throws URISyntaxException { HttpClient httpClient = HttpClient .newBuilder() .connectTimeout(Duration.ofMinutes(2)) .build(); List\u0026lt;CompletableFuture\u0026lt;HttpResponse\u0026lt;String\u0026gt;\u0026gt;\u0026gt; list = requests .stream() .map(request -\u0026gt; httpClient.sendAsync(request, HttpResponse.BodyHandlers.ofString())) .map(future -\u0026gt; future.thenApplyAsync(response -\u0026gt; { System.out.println(\u0026#34;Thread \u0026#34; + Thread.currentThread().getName() + \u0026#34; current time \u0026#34; + System.currentTimeMillis()); return response; })).collect(Collectors.toList()); return list.stream() .map(future -\u0026gt; future.exceptionally(throwable -\u0026gt; null)) .map(CompletableFuture::join) .collect(Collectors.partitioningBy(httpResponse -\u0026gt; httpResponse.statusCode() == 200)); } After getting the map and extracting the list out of it we can check HttpResponse.body for the response. For this instance its a String output for chuck norris api\u0026rsquo;s joke.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 class Day53 { public static void main(String[] args) throws URISyntaxException, ExecutionException, InterruptedException { String apiURL = \u0026#34;https://api.chucknorris.io/jokes/random\u0026#34;; String method = \u0026#34;GET\u0026#34;; List\u0026lt;HttpRequest\u0026gt; httpRequests = getHttpRequests(apiURL,3); HttpClient httpClient = HttpClient .newBuilder() .connectTimeout(Duration.ofMinutes(2)) .build(); Map\u0026lt;Boolean, List\u0026lt;HttpResponse\u0026lt;String\u0026gt;\u0026gt;\u0026gt; result = httpRequestDispatcher(httpRequests); List\u0026lt;HttpResponse\u0026lt;String\u0026gt;\u0026gt; success = result.get(true); List\u0026lt;HttpResponse\u0026lt;String\u0026gt;\u0026gt; failed = result.get(false); System.out.println(\u0026#34;Success\u0026#34;); success.forEach(httpResponse -\u0026gt; System.out.println(httpResponse.body())); System.out.println(\u0026#34;Failed\u0026#34;); failed.forEach(httpResponse -\u0026gt; System.out.println(httpResponse.body())); } static List\u0026lt;HttpRequest\u0026gt; getHttpRequests(String url, int requestInstances) { return IntStream.range(0,requestInstances).mapToObj(value -\u0026gt; { try { return HttpRequest .newBuilder() .uri(new URI(url)) .GET() .build(); } catch (URISyntaxException e) { throw new RuntimeException(e); } }).collect(Collectors.toList()); } } After getting all the requests sorted into success and failed list we can check out the body of the requests for the joke and the error response the API returns.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day53/","summary":"Using CompletableFuture\u0026rsquo;s execution to process http request asynchronously","title":"Day 53: Processing Http Request Asynchronously using CompletableFutures"},{"content":"What is Synchronous and Asynchronous: Java code executes line by line.\u0026quot;Synchronously\u0026quot; means \u0026ldquo;using the same clock\u0026rdquo; so when two instructions are synchronous they use the same clock and must happen one after the other. \u0026ldquo;Asynchronous\u0026rdquo; means \u0026ldquo;not using the same clock\u0026rdquo; so the instructions are not concerned with being in step with each other. That\u0026rsquo;s why it looks backwards, the term is not referring to the instructions relationship to each other. It\u0026rsquo;s referring to each instructions relationship to the clock\nThere are a couple of ways in which you can make asynchronous calls in Java depending upon your requirement. (But controlling them well is extremely important and tricky at times.)\nIn addition to Runnable interface (which is used to just execute tasks but doesn\u0026rsquo;t return anything to caller) ,you can read about Callable interface and Future objects in java. (Which can return data to the caller)\nCompletionService: A CompletionService can be used to manage asynchronous IO, in which tasks that perform reads are submitted in one part of a program or system, and then acted upon in a different part of the program when the reads complete, possibly in a different order than they were requested(in the order of completion).\nCompletionService manages an internal completion queue.\nLet’s understand in layman language.\nSuppose you want to execute n number of tasks in parallel, you will think of using threads, now how will you manage all threads execution? okay one may say with the help of Executor Service we can handle those threads. Completion service also solves the same thing for you but gives you an advantage when tasks are completed.\nWith ExecutorService, once you have submitted the tasks to run, you need to manually code for efficiently getting the results of the tasks completed.\nWith CompletionService, this is pretty much automated. Imagine you have a list of tasks to be subdraftmitted. Then, instead of trying to find out which task has completed (to get the results), it just asks the CompletionService instance to return the results as they become available.\nCompletion service provides functions to get completed tasks from its internal queue in the order they have completed.\nExecutorCompletionService: A CompletionService that uses a supplied Executor to execute tasks. This class arranges that submitted tasks are, upon completion, placed on a queue accessible using take. The class is lightweight enough to be suitable for transient use when processing groups of tasks.\nUsing a ExecutorCompletionService to call http get requests to a jokes api to get response concurrently and completes each request as it completes and then we poll from the service to take the outputs of the get request.\nFirstly creating a custom HttpCallable class to send, process http get response\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 class HttpCallable implements Callable\u0026lt;String\u0026gt; { HttpClient httpClient; HttpRequest httpRequest; HttpCallable(HttpClient httpClient, HttpRequest request) { this.httpClient = httpClient; this.httpRequest = request; } public String call() throws IOException, InterruptedException { return httpClient.send(httpRequest, HttpResponse.BodyHandlers.ofString()).body()+ \u0026#34;\\n\u0026#34; + \u0026#34;Thread \u0026#34; + Thread.currentThread().getName() + \u0026#34; current time \u0026#34; + System.currentTimeMillis(); } } this classes call() method executes httpclient sendAsync method, which takes HttpRequest object. httpClient.send().body() returns a string body as the response is handled with BodyHandlers.ofString()\nTo create the Http requests.\n1 2 3 4 5 6 7 8 9 10 11 12 13 static List\u0026lt;HttpRequest\u0026gt; getHttpRequests(String url, int requestInstances) { return IntStream.range(0,requestInstances).mapToObj(value -\u0026gt; { try { return HttpRequest .newBuilder() .uri(new URI(url)) .GET() .build(); } catch (URISyntaxException e) { throw new RuntimeException(e); } }).collect(Collectors.toList()); } Now to use the callables and submit to ExecutorCompletionService\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 static void httpDispatcherExecutionCompletion(HttpClient httpClient, List\u0026lt;HttpRequest\u0026gt; requests) { final ExecutorService pool = Executors.newFixedThreadPool(3); final CompletionService\u0026lt;String\u0026gt; service = new ExecutorCompletionService\u0026lt;\u0026gt;(pool); final List\u0026lt;? extends Callable\u0026lt;String\u0026gt;\u0026gt; callables = requests.stream().map(request -\u0026gt; new HttpCallable(httpClient,request)).collect(Collectors.toList()); for (final Callable\u0026lt;String\u0026gt; callable : callables) { service.submit(callable); } pool.shutdown(); try { while (!pool.isTerminated()) { final Future\u0026lt;String\u0026gt; future = service.take(); System.out.println(future.get()); } } catch (ExecutionException | InterruptedException ex) { } } to process the request\u0026rsquo;s first create a ExecutorService. Passing that pool to ExecutorCompletionService. Then taking each Callables and submitting them to the ExecutorCompletionService to process. As the pool will terminate after processing all callables I check that and take Future response from the inner queue of ExecutionCompleitonService and then get the value.\nNow to connect it all\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 class Day52 { public static void main(String[] args) throws URISyntaxException, ExecutionException, InterruptedException { String apiURL = \u0026#34;https://api.chucknorris.io/jokes/random\u0026#34;; String method = \u0026#34;GET\u0026#34;; List\u0026lt;HttpRequest\u0026gt; httpRequests = getHttpRequests(apiURL,3); HttpClient httpClient = HttpClient .newBuilder() // .executor(Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors())) .connectTimeout(Duration.ofMinutes(2)) .build(); System.out.println(\u0026#34;---------------------- ExecutorCompletionService-------------\u0026#34;); httpDispatcherExecutionCompletion(httpClient, httpRequests); } } In the next part will do the exact thing using CompletableFuture and its functionality to asynchronous execution of http get requests.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day52/","summary":"Async Processing HttpRequest\u0026rsquo;s using ExecutorCompletionService","title":"Day 52: Async Processing HttpRequest's using ExecutorCompletionService"},{"content":"Creating a Stream using Java StreamSupport api and iterator, spliterator and then partition the stream using Collections.partitionBy() and Collections.groupBy() method.\nStream: Introduced in Java 8, the Stream API is used to process collections of objects. A stream is a sequence of objects that supports various methods which can be pipelined to produce the desired result. The features of Java stream are –\nA stream is not a data structure instead it takes input from the Collections, Arrays or I/O channels. Streams don’t change the original data structure, they only provide the result as per the pipelined methods. Each intermediate operation is lazily executed and returns a stream as a result, hence various intermediate operations can be pipelined. Terminal operations mark the end of the stream and return the result. To make a Stream from Iterator We first need to make a Spliterator first. So lets first understand what is Iterator and Spliterator is.\nIterator: A Java Cursor is an Iterator, which is used to iterate or traverse or retrieve a Collection or Stream object’s elements one by one. There are three cursors in Java. Spliterator is also a special type of Iterator so its also a cursor.\nIterator Enumeration ListIterator Spliterator Iterators in Java are used in the Collection framework to retrieve elements one by one lazily. It is a universal iterator as we can apply it to any Collection object. By using Iterator, we can perform both read/update and remove operations. It is an improved version of Enumeration with the additional functionality of removing an element.\nIterator must be used whenever we want to enumerate elements in all Collection framework implemented interfaces like Set, List, Queue, Deque, and all implemented classes of Map interface.\nSpliterator: Besides traversing sequences of data, like an Iterator\u0026lt;T\u0026gt; a Spliterator\u0026lt;T\u0026gt; can also partition it:\nIterator + Splitting =\u0026gt; Spliterator\nThe trySplit() method allows it to partition off some elements of the sequence as another Spliterator\u0026lt;T\u0026gt;.\nThis particular advantage over Iterator makes it the core component of the Stream API. By splitting up data into apt sub-sequences, it allows parallel processing.\nIterator to Spliterator: To convert an Iterator to Spliterator there is two api\u0026rsquo;s. One is for unknownsized collection and another is for known sized collections. In the known sized Spliterators.spliterator() requires the size whereas the unknownsized doesnt required the size.\nThere is another parameter that is needed to be provided to the method, and that is characterstics, which is an integer that defines the characterstics of the spliterator.\nType of spliterator\u0026rsquo;s\nEmpty Spliterator [Spliterators.emptySpliterator();] Sized Spliterator [Spliterators.spliterator();] Unknownsized Spliterator [Spliterators.spliteratorUnknownSize();] Now creating a unknownsized spliterator from and Iterator\n1 2 Iterator\u0026lt;Node\u0026gt; nodeIterator = Iterators.nodes(result); Spliterator\u0026lt;Node\u0026gt; nodeSpliterator = Spliterators.spliteratorUnknownSize(nodeIterator,Spliterator.CONCURRENT); Now StreamSupport.stream() method can take that Spliterator Object and a boolean option to enable parallel processing of stream elements. This method Creates a new sequential or parallel Stream from a Spliterator.\n1 Stream\u0026lt;Node\u0026gt; nodeStream = StreamSupport.stream(nodeSpliterator,false); The spliterator is only traversed, split, or queried for estimated size after the terminal operation of the stream pipeline commences.\nIf parallel option is enabled then the returned stream will be parallel and if false then it will return a sequential stream.\nThis is a great stackoverflow article to understand if we should use parallel everywhere possible?\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day51/","summary":"Creating Stream using StreamSupport and splitting it using partitioning and grouping","title":"Day 51: Creating Stream (using StreamSupport API)  from Spliterator by converting Iterator to Spliterator"},{"content":"At first lets understand what is an Interface in Java\nInterface: An Interface in Java programming language is defined as an abstract type used to specify the behavior of a class. An interface in Java is a blueprint of a class. A Java interface contains static constants and abstract methods and from java 8 default implementations.\nThe interface in Java is a mechanism to achieve abstraction. There can be abstract methods in the Java interface, and method body for default methods. It is used to achieve abstraction and multiple inheritance in Java. In other words, you can say that interfaces can have abstract methods and variables. Java Interface also represents the IS-A relationship.\nMarker Interface: Market Interface is an interface that has no method declarations or fields in it. It is used as a tag to let the compiler know it needs to add some special behavior to the class implementing the marker interface. That is why the marker interface is also known as the tag interface in Java. Some java interfaces have no members like\njava. lang.Cloneable java.io.Serializable\nthese both interfaces are Marker Interfaces because it has no members. Marker interfaces are used to mark a class for a purpose. Purpose does not require any additional functionality.\nUse of Clonable \u0026amp; Serialization Marker interfaces are shown below.\nFor Example java.io.Serializable interface is defined into java.io classes\nObjectInputStream ObjectOutputStream\nMarker Interfaces provide runtime information about Object. So the compiler and the JVM have additional information about the Object.\nAs we can see clonable and serialization implementes marker type interface so lets see example of both\nClonable: Cloneable interface is implemented by a class to make Object.clone() method valid thereby making field-for-field copy. This interface allows the implementing class to have its objects to be cloned instead of using a new operator.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 class Example implements Cloneable { int number; String str; // Example class constructor public Example(int i, String s) { this.number = number; this.str = str; } // Overriding clone() method // by simply calling Object class // clone() method. @Override protected Object clone() throws CloneNotSupportedException { return super.clone(); } } public class Day50 { public static void main(String[] args) throws CloneNotSupportedException { Example anotherExample = new Example(20, \u0026#34;Example\u0026#34;); // cloning \u0026#39;anotherExample\u0026#39;is holding // new cloned object reference in example // down-casting as clone() return type is Object Example example = (Example) anotherExample.clone(); System.out.println(example.number); System.out.println(example.str); } } Serializable: It is a marker interface in Java that is defined in the java.io package. If we want to make the class serializable, we must implement the Serializable interface. If a class implements the Serializable interface, we can serialize or deserialize the state of an object of that class.\nSerialization: it is the action of converting an object into a byte stream. A mechanism in which the object state is read from the memory and written into a file or database or pass over the network. Deserialization (converting byte stream into an object) is the opposite of serialization means that object state reading from a file , database or over the network sending request and response and written back into memory is called deserialization of object.\nSerialization (writing) can be achieved with the ObjectOutputStream class and deserialization (reading) can be achieved with the ObjectInputStream class.\nExample:\n1 2 3 4 5 6 7 8 9 10 11 12 import java.io.Serializable; public class User implements Serializable { int id; String name; public User(int id, String name) { this.id = id; this.name = name; } } now to write the user object in a file / database by serializing the data.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 public class Day50 { public static void main(String args[]) { try { //Creating the object User user = new User(000001,\u0026#34;Kasun\u0026#34;); //Creating stream and writing the object FileOutputStream fos=new FileOutputStream(\u0026#34;Users.txt\u0026#34;); ObjectOutputStream out=new ObjectOutputStream(fos); out.writeObject(user); out.flush(); //closing the stream out.close(); System.out.println(\u0026#34;Data has been written to the file.\u0026#34;); } catch(Exception e) { e.printStackTrace(); } } } now to deserialize the object from the file\n1 2 3 4 5 6 7 8 9 10 11 12 public class Day50 { public static void main(String[] args) throws IOException, ClassNotFoundException { FileInputStream fis = new FileInputStream(\u0026#34;Users.txt\u0026#34;); ObjectInputStream ois = new ObjectInputStream(fis); User user = (User)ois.readObject(); System.out.println(\u0026#34;User Name: \u0026#34;+ user.name); System.out.println(\u0026#34;User ID: \u0026#34;+ user.id); ois.close(); } } reading data and deserialize the data to a User object from the Users.txt file.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day50/","summary":"Marker interface and it\u0026rsquo;s uses","title":"Day 50: Marker interface and it's uses"},{"content":"In previous post we have explored and wrote a utility class for zipping and unzipping files and folders using java Zip api. Now we will see another api provided by java named FileSystem under java.nio.file.\nFileSystem: FileSystem api of java non-blocking io\u0026rsquo;s file is an interface to underlying file system. It\u0026rsquo;s the factory for object to access File and other objects in the file system.\nThe default file system, obtained by invoking the FileSystems.getDefault() method, provides access to the file system that is accessible to the Java virtual machine. The FileSystems class defines methods to create file systems that provide access to other types of (custom) file systems.\nIn java there are 3 file system providers.\nA provider is a factory for one or more FileSystem instances. Each file system is identified by a URI where the URI\u0026rsquo;s scheme matches the provider\u0026rsquo;s scheme. The default file system, for example, is identified by the URI \u0026quot;file:///\u0026quot;. A memory-based file system, for example, may be identified by a URI such as \u0026quot;memory:///?name=logfs\u0026quot;. The newFileSystem method may be used to create a file system, and the getFileSystem method may be used to obtain a reference to an existing file system created by the provider. Where a provider is the factory for a single file system then it is provider dependent if the file system is created when the provider is initialized, or later when the newFileSystem method is invoked. In the case of the default provider, the FileSystem is created when the provider is initialized.\nFileSystemProviders: A Service-provider class for file systems. A file system provider is a concrete implementation of this class that implements the abstract methods defined by this class. A provider is identified by a URI scheme. The default provider is identified by the URI scheme \u0026ldquo;file\u0026rdquo;. It creates the FileSystem that provides access to the file systems accessible to the Java virtual machine. The FileSystems class defines how file system providers are located and loaded. The default provider is typically a system-default provider but may be overridden if the system property java.nio.file.spi.DefaultFileSystemProvider is set. In that case, the provider has a one argument constructor whose formal parameter type is FileSystemProvider. All other providers have a zero argument constructor that initializes the provider.\nZipFileSystemProvider: The zip file system provider treats a zip or JAR file as a file system and provides the ability to manipulate the contents of the file. The zip file system provider creates multiple file systems — one file system for each zip or JAR file.\nZipFileSystemProvider to create a Folder.zip and archive a single file and a direcotry/folder\nSingle File to Zip file 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 public static void ZipSingleFile(Path source, String fileName) { if (!Files.isRegularFile(source)) { System.err.println(\u0026#34;Please provide a file.\u0026#34;); return; } Map\u0026lt;String, String\u0026gt; env = new HashMap\u0026lt;\u0026gt;(); // Create the zip file if it doesn\u0026#39;t exist env.put(\u0026#34;create\u0026#34;, \u0026#34;true\u0026#34;); URI uri = URI.create(\u0026#34;jar:file:/home/mohibulhasan/Documents/\u0026#34; + fileName); try (FileSystem zipFileSystem = FileSystems.newFileSystem(uri, env)) { Path pathInZipfile = zipFileSystem.getPath(source.getFileName().toString()); // Copy a file into the zip file path Files.copy(source, pathInZipfile, StandardCopyOption.REPLACE_EXISTING); } catch (IOException e) { throw new RuntimeException(e); } } Folder to Zip: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 public static void zipFolder(Path source) throws IOException { // get current working directory String currentPath = System.getProperty(\u0026#34;user.dir\u0026#34;) + File.separator; // get folder name as zip file name // can be other extension, .foo .bar .whatever String zipFileName = source.getFileName().toString() + \u0026#34;.zip\u0026#34;; URI uri = URI.create(\u0026#34;jar:file:\u0026#34; + currentPath + zipFileName); Files.walkFileTree(source, new SimpleFileVisitor\u0026lt;\u0026gt;() { @Override public FileVisitResult visitFile(Path file, BasicFileAttributes attributes) { // don\u0026#39;t copy symboliclink as its not supported if (attributes.isSymbolicLink()) { return FileVisitResult.CONTINUE; } Map\u0026lt;String, String\u0026gt; env = new HashMap\u0026lt;\u0026gt;(); env.put(\u0026#34;create\u0026#34;, \u0026#34;true\u0026#34;); try (FileSystem zipfs = FileSystems.newFileSystem(uri, env)) { Path targetFile = source.relativize(file); Path pathInZipfile = zipfs.getPath(targetFile.toString()); // NoSuchFileException, need create parent directories in zip path if (pathInZipfile.getParent() != null) { Files.createDirectories(pathInZipfile.getParent()); } // copy file attributes CopyOption[] options = {StandardCopyOption.REPLACE_EXISTING, StandardCopyOption.COPY_ATTRIBUTES, LinkOption.NOFOLLOW_LINKS}; // Copy a file into the zip file path Files.copy(file, pathInZipfile, options); } catch (IOException e) { e.printStackTrace(); } return FileVisitResult.CONTINUE; } @Override public FileVisitResult visitFileFailed(Path file, IOException exc) { System.err.printf(\u0026#34;Unable to zip : %s%n%s%n\u0026#34;, file, exc); return FileVisitResult.CONTINUE; } }); } In this case by accessing and then copying the file from the Folder.zip file to and external folder will behave like unzip.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day49/","summary":"Zip and Unzip file using Java FileSystem api","title":"Day 49: A utility class for Zip and Unzip files using java FileSystem api (Part 2)"},{"content":"In java there is api for Zip (archiving) and Unzip a file or a set of files and directory and then place its contents or the zip file in the directory.\nIn java there are two ways of zipping and unzipping file\nUsing ZipInputStream and ZipOutputStream with ZipEntry Using Zip File System Provider For part 1 i will write a utility class using ZipInputStream and ZipOutputStream class to zip and unzip file and folder. In the subsequent part i will write about zip file system provider api\nWhat is an archive in context of computer filesystem : an archive file is a computer file that is composed of one or more files along with metadata. Archive files are used to collect multiple data files together into a single file for easier portability and storage, or simply to compress files to use less storage space.\nZIP File: ZIP is an archive file format that supports lossless data compression.\nA ZIP file may contain one or more files or directories that may have been compressed. The ZIP file format permits a number of compression algorithms.\nIn java when used ZipInputStream and ZipOutputStream we can create zip file and by using ZipEntry to entry the file in the .zip file. To better understand ZipEntry a picture of files and its zip file and what are ZipEntry regarding that file is given below.\nZip Single File: Now to create a zip file of any particular file an utility class is created ZipUtils which will use ZipInputStream along with ZipOutputStream to zip and unzip any files. In this ZipUtils class the zipSingleFile() method will take the file path for\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 public static class ZipUtils { public static void zipSingleFile(Path source, Path zipFileName) throws IOException { if (!Files.isRegularFile(source)) { System.err.println(\u0026#34;Please provide a file.\u0026#34;); return; } try (ZipOutputStream zos = new ZipOutputStream(new FileOutputStream(zipFileName.toString())); FileInputStream fis = new FileInputStream(source.toFile())) { ZipEntry zipEntry = new ZipEntry(source.getFileName().toString()); zos.putNextEntry(zipEntry); byte[] buffer = new byte[1024]; int len; while ((len = fis.read(buffer)) \u0026gt; 0) { zos.write(buffer, 0, len); } zos.closeEntry(); } } } this zipSingleFile() takes two arguments one is the source file path and another is for where to store the created zip file and its name.\n1 2 3 4 5 6 7 8 class Day48 { public static void main(String[] args) throws IOException { final Path OUTPUT_ZIP_FILE = Path.of(\u0026#34;/home/mohibulhasan/Documents/Folder.zip\u0026#34;); final Path SOURCE_FILE = Path.of(\u0026#34;/home/mohibulhasan/Documents/example.txt\u0026#34;); ZipUtils.zipSingleFile(Path.of(\u0026#34;/home/mohibulhasan/Documents/example.txt\u0026#34;),OUTPUT_ZIP_FILE); } } in this case the new zip file that will be created would be inside /home/mohibulhasan/Documents/ and as Folder.zip\nZip a folder: In case of a directory in ZipUtils class there is a method zipFolder() which will use a source directory as path to zip all the contents of that directory\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 public static void zipFolder(Path source) throws IOException { // get folder name as zip file name String zipFileName = source.toFile().getAbsolutePath() + \u0026#34;.zip\u0026#34;; try (ZipOutputStream zos = new ZipOutputStream(new FileOutputStream(zipFileName))) { Files.walkFileTree(source, new SimpleFileVisitor\u0026lt;\u0026gt;() { @Override public FileVisitResult visitFile(Path file, BasicFileAttributes attributes) { // only copy files, no symbolic links if (attributes.isSymbolicLink()) { return FileVisitResult.CONTINUE; } try (FileInputStream fis = new FileInputStream(file.toFile())) { Path targetFile = source.relativize(file); zos.putNextEntry(new ZipEntry(targetFile.toString())); byte[] buffer = new byte[1024]; int len; while ((len = fis.read(buffer)) \u0026gt; 0) { zos.write(buffer, 0, len); } // if large file, throws out of memory //byte[] bytes = Files.readAllBytes(file); //zos.write(bytes, 0, bytes.length); zos.closeEntry(); System.out.printf(\u0026#34;Zip file : %s%n\u0026#34;, file); } catch (IOException e) { e.printStackTrace(); } return FileVisitResult.CONTINUE; } @Override public FileVisitResult visitFileFailed(Path file, IOException exc) { System.err.printf(\u0026#34;Unable to zip : %s%n%s%n\u0026#34;, file, exc); return FileVisitResult.CONTINUE; } }); } } in this method Files.walkFileTree() is used for to visit all the files in the directory and take each file and put them inside the zipEntry. Then ZipOutputStream is used to create the zipEntry\u0026rsquo;s and the added.\nUnzip a .zip File: If need to unzip a .zip file and extract the files and folder of that zip file, we need to give the zip file path and destiantion path where the .zip file will be extracted.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 public static void unzipIt(Path sourceFilePath, Path destFilePath) { try (ZipInputStream zis = new ZipInputStream(new FileInputStream(sourceFilePath.toFile()))) { // list files in zip ZipEntry zipEntry = zis.getNextEntry(); while (zipEntry != null) { boolean isDirectory = false; // some zip stored files and folders separately // e.g data/ // data/folder/ // data/folder/file.txt if (zipEntry.getName().endsWith(File.separator)) { isDirectory = true; } Path newPath = zipSlipProtect(zipEntry, destFilePath); if (isDirectory) { Files.createDirectories(newPath); } else { // some zip stored file path only, need create parent directories // e.g data/folder/file.txt if (newPath.getParent() != null) { if (Files.notExists(newPath.getParent())) { Files.createDirectories(newPath.getParent()); } } // copy files, nio Files.copy(zis, newPath, StandardCopyOption.REPLACE_EXISTING); // copy files, classic /*try (FileOutputStream fos = new FileOutputStream(newPath.toFile())) { byte[] buffer = new byte[1024]; int len; while ((len = zis.read(buffer)) \u0026gt; 0) { fos.write(buffer, 0, len); } }*/ } zipEntry = zis.getNextEntry(); } zis.closeEntry(); } catch (IOException e) { } } while doing the unzip all the zipEntry are checked and until all of them are processed the mehtod will check if that zip entry is a file or a folder and if its a folder it will create a folder in the desitantion folder for that particular entry. Some file will have relative path so it will also make the parent folders.\nWhile trying to unzip a .zip in this case the zip file it may cause ZipSlip Attack,\nZip Slip Attack: The zip slip attack is an attack that adds entries to a zip file that will be unzipped, entries consisting relative file paths with one or more /.. sections in the path the final path of the file could end up being outside the directory into which the ZipFile is requested unzipped to. Let\u0026rsquo;s look at an example:\nZip file to be unzipped to the directory /apps/example/data/unzipped-file. An entry in the Zip file has the relative path ../../../../etc/hosts . The final path of that entry becomes: /apps/example/data/unzipped-file/../../../../etc/hosts which is equivalent of /etc/hosts .\nIn the case of linux based systems unzipping this file could potentially overwrite our hosts file,enabling the attacker to point e.g. www.facebook.com to an IP address of their own choice. The next time you try to access Facebook from that computer, it won\u0026rsquo;t be the real Facebook you are accessing, but the attacker\u0026rsquo;s spoofed version. Once you login, the attacker now has your username and password, and your Facebook account can be hacked.\nTo avoid this a method zipSlipProtect() method is implemented which will check if the path of the zip entry is outside of the destination directory or not.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 private static Path zipSlipProtect(ZipEntry zipEntry, Path destinationDirectory) throws IOException { // test zip slip vulnerability // Path targetDirResolved = targetDir.resolve(\u0026#34;../../\u0026#34; + zipEntry.getName()); Path targetDirResolved = destinationDirectory.resolve(zipEntry.getName()); // make sure normalized file still has targetDir as its prefix // else throws exception Path normalizePath = targetDirResolved.normalize(); if (!normalizePath.startsWith(destinationDirectory)) { throw new IOException(\u0026#34;Bad zip entry: \u0026#34; + zipEntry.getName()); } return normalizePath; } in this method by using the Path.resolve() and Path.normalize() to check if the normalize file path still has the destiantion direcotry as its prefix or not.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day48/","summary":"A utility class to zip and unzip files and directories","title":"Day 48: A utility class for Zip and Unzip files using java Zip api (Part 1)"},{"content":"Java Management Extension or JMX framwork provides an easily configurable, scalable, reliable and more or less friendly infrastructure for managing Java application either locally or remotely. The framework introduces the concept of MBeans for real-time management of applications.\nMXBean: An MXBean is a type of MBean that references only a predefined set of data types. In this way, you can be sure that your MBean will be usable by any client, including remote clients, without any requirement that the client have access to model-specific classes representing the types of your MBeans\nThreadMXBean: The management interface for the thread system of the Java virtual machine.\nA Java virtual machine has a single instance of the implementation class of this interface. This instance implementing this interface is an MXBean that can be obtained by calling the ManagementFactory.getThreadMXBean() method or from the platform MBeanServer method.\nUsing ThreadMXBean we can get information about\nDumping all live threads and their stack trace Finding deadlock threads those are waiting to get monitors / ownable synchronizes Thread count for live daemon / non daemon threads If cpu thread time and user thread time and Thread Contention Monitoring supported So now using ThreadMXBean get thread count, currentThreadCpuTime, currentThreadUserTime\n1 2 3 4 5 6 7 8 9 10 11 12 13 class Day47 { public static void main(String[] args) { ThreadMXBean threadMXBean = ManagementFactory.getThreadMXBean(); System.out.println(\u0026#34;Thread count is \u0026#34;+threadMXBean.getThreadCount()); System.out.println(\u0026#34;Current Thread CPU time \u0026#34;+TimeUnit.MILLISECONDS.convert(threadMXBean.getCurrentThreadCpuTime(),TimeUnit.NANOSECONDS)); System.out.println(MessageFormat.format(\u0026#34;Current Thread User time {0} \u0026#34;,TimeUnit.MILLISECONDS.convert(threadMXBean.getCurrentThreadUserTime(),TimeUnit.NANOSECONDS)); List\u0026lt;ThreadInfo\u0026gt; threadInfos = List.of(threadMXBean.dumpAllThreads(true, true)); for (ThreadInfo threadInfo: threadInfos) { System.out.println(MessageFormat.format(\u0026#34;Thread Name is {0} | Thread id -\u0026gt; {1}\u0026#34;,threadInfo.getThreadName(),threadInfo.getThreadId())); } } } Here getting total thread count running in jvm, current thread cpu time (if JVM supports it) it returns in nano second so using TimeUnit.MILISECONDS.convert() to convert it to Miliseconds also current thread user time. After that dumping all the threads and getting the name and id of the threads.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day47/","summary":"Using ThreadMXBean get thread execution and monitoring related information for running threads in jvm","title":"Day 47: ThreadMXBean to get information about thread's status, deadlock monitoring and cpu time that the thread got etc."},{"content":"When objects are expensive to create and they are needed only for short periods of time it is advantageous to utilize the Object Pool pattern. The Object Pool provides a cache for instantiated objects tracking which ones are in use and which are available.\nHere creating a HttpClient object pool to create HttpClientPool to get avaialbe and in use instances\nTo do that at first created a generic ObjectPool\u0026lt;T\u0026gt; where to create an Object pool we have to pass the type of the object and then extend a class to implement the abstraced behavior of the Object pools createing instance method create().\ncreate() method will create the instance and there are two sets to manage the instances. available set stores currently available instances from the pool and in use set stores in uses instances of the pool. By calling the getInstance() and releaseInstance() method we can get and release an instance to object pool.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 abstract class ObjectPool\u0026lt;T\u0026gt; { private final Set\u0026lt;T\u0026gt; available = new HashSet\u0026lt;\u0026gt;(); private final Set\u0026lt;T\u0026gt; inUse = new HashSet\u0026lt;\u0026gt;(); protected abstract T create(); public synchronized T getInstance() { if (available.isEmpty()) { available.add(create()); } var instance = available.iterator().next(); available.remove(instance); inUse.add(instance); return instance; } public synchronized void releaseInstance(T instance) { inUse.remove(instance); available.add(instance); } @Override public synchronized String toString() { return String.format(\u0026#34;Pool available=%d inUse=%d\u0026#34;, available.size(), inUse.size()); } } when extending this abstract class, creating process should be implemented in the create() method.\n1 2 3 4 5 6 class HttpClientPool extends ObjectPool\u0026lt;HttpClient\u0026gt; { @Override protected HttpClient create() { return HttpClient.newHttpClient(); } } to see how the ObjectPool works now we use it in a class.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 class Day46 { public static void main(String[] args) { HttpClientPool httpClientPool = new HttpClientPool(); System.out.println(httpClientPool.toString()); HttpClient httpClient1 = httpClientPool.getInstance(); String releaseInstance = \u0026#34;Release Instance \u0026#34;; String getInstance = \u0026#34;Get Instance \u0026#34;; System.out.println(getInstance+ httpClient1); System.out.println(httpClientPool.toString()); HttpClient httpClient2 = httpClientPool.getInstance(); System.out.println(getInstance+ httpClient2); System.out.println(httpClientPool.toString()); HttpClient httpClient3 = httpClientPool.getInstance(); System.out.println(getInstance+httpClient3); System.out.println(httpClientPool.toString()); System.out.println(releaseInstance+httpClient1); httpClientPool.releaseInstance(httpClient1); System.out.println(httpClientPool.toString()); System.out.println(releaseInstance+httpClient2); httpClientPool.releaseInstance(httpClient2); System.out.println(httpClientPool.toString()); HttpClient httpClient4 = httpClientPool.getInstance(); System.out.println(getInstance+ httpClient4); System.out.println(httpClientPool.toString()); HttpClient httpClient5 = httpClientPool.getInstance(); System.out.println(getInstance+ httpClient5); System.out.println(httpClientPool.toString()); } } here the output is\nGet Instance jdk.internal.net.http.HttpClientImpl@75c072cb(1)\rPool available=0 inUse=1\rGet Instance jdk.internal.net.http.HttpClientImpl@e50a6f6(2)\rPool available=0 inUse=2\rGet Instance jdk.internal.net.http.HttpClientImpl@53ca01a2(3)\rPool available=0 inUse=3\rRelease Instance jdk.internal.net.http.HttpClientImpl@75c072cb(1)\rPool available=1 inUse=2\rRelease Instance jdk.internal.net.http.HttpClientImpl@e50a6f6(2)\rPool available=2 inUse=1\rGet Instance jdk.internal.net.http.HttpClientImpl@e50a6f6(2)\rPool available=1 inUse=2\rGet Instance jdk.internal.net.http.HttpClientImpl@75c072cb(1)\rPool available=0 inUse=3 at first pool does not have any instances then it creates and tag it as a in use instance1. then it does the same for instance2 and instance3 then it releases instance and thus the available set shows it have one instance then instance2 is released\nso inuse = 1 and available = 2 and then again calling etInstance() gives instance from the available set, thus not creating any new instances of HttpClient\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day46/","summary":"Implementing ObjectPool pattern to get any type object pooling","title":"Day 46: Writing a generic and thread safe ObjectPool to enable pooling for any type of class"},{"content":"Runtime and Runtime Environment are some of the most overloaded terms in software development. It’s confusing for everyone; this word means many different things in many different contexts. This post’s goal is to provide you with an intuition behind the many use-cases of Runtime\nRuntime is used both as adjective and noun in most of the time.\nRuntime — The Lifecycle (Noun) The first meaning of runtime is with regards to program lifecycle. This refers to the period of time in which a program is executing. A intuitive comparison is often made between compile time and runtime.\nThe IDE is spitting out a lot of new warnings during compile time.\nThis process is outputting a lot of logs during runtime.\nRuntime — During The Lifecycle (Adjective) In the same context of program lifecycle, runtime is also commonly used as an adjective. This is sometimes hyphenated as “run-time.” The O’Reilly Style Guide recommends using “runtime” for both nouns and adjectives.\nRuntime Environment Armed with an intuition of runtime as a lifecycle, let’s discuss runtime environments. These two concepts are related, but subtlety different. Confusion is created because people abridge “runtime environment” to just “runtime.”\nYou have to be careful that your Javascript program doesn’t have any runtime errors when it’s running in the Node.js runtime [environment].\nQ: Did you see the release notes for the new JRE?\nA: Yea, there’s a lot of fancy stuff going on in the new Java Runtime [Environment].\nIn these examples, the standalone “runtime” refers to a “runtime environment.” In these situations, it’s better to be precise with your language.\nThe new Java Runtime Environment seems to be doing a better job with garbage collection.\nIs there a way to control how many threads handle the I/O within the Node.js runtime environment?\nUnderstanding Runtime by Cooking a Pizza After developing a hunger for a chicken cheese pizza, you decide to hop on Google to look for a recipe. After sifting through a few search results and scrolling past a few walls of ads, you find a clean step-by-step recipe that seems like a winner.\nUnfortunately, a recipe on its own is useless. The recipe you found on Google isn’t actually going to get you your pizza, you need a kitchen! The pizza recipe’s runtime environment is your kitchen.\nYour code is just code. Whatever code you write, in whatever language you choose, needs to eventually execute on a computer. Runtime environments enable this execution.\nCommon Runtime Environment ( The Operating System ) The universal runtime environment for any kind of programmatic execution is the operating system. The operating system is the only way you can get the CPU to execute your code. The OS is the silent hero that ensures your program gets some memory, gets scheduled fairly, and doesn’t disturb its neighbors. It doesn’t matter if you’re using C, Python, or Node.js—at the end of the day, the operating system is everyone’s runtime environment.\nExecutable File format Every operating system defines a binary file format for executable code. In Unix-Like operating systems, this is the ELF file format (.so) . For windows its executable (.exe)\nExecutable\u0026rsquo;s Runtime Environment An executable’s runtime environment is the operating system. In addition, operating systems ship with programs that are able to take these files and give them to the hardware to be executed. In Linux, one example is the execve() program. For all other operating systems, there will be a similar set of file formats and loader programs.\nHigher-Level Runtime Environments As programming languages evolved, people wanted an environment that could handle additional tasks that felt cumbersome for developers.Wouldn’t some kind of automatic reference counting and garbage collection be extremely convenient? Some people say yes and this is how fancier runtime environments like the JRE developed.\nAnother popular higher-level runtime environment is Node.js. Many developers refer to this as the “Node.js Runtime” or simply just “Node.” Node is a runtime environment for the Javascript language, similar to how the JRE is a runtime environment for the Java language. Node comes with fancy features like a callback queue, an event loop, and a thread pool. Just like the JRE, these bells and whistles exist to make our lives easier as developers.\nWith higher-level runtime environments, features can become blurred across the language and its environment. Within the Java ecosystem, the automatic garbage collection feature is not technically a feature of the Java language itself, but is actually a feature of the JRE. 95% of developers won’t bother making this distinction. The statement “Java has automatic memory management” can more precisely be stated as, “The Java Runtime Environment has automatic memory management.” As a warning, you’ll get some unpleasant glares if you ever decide to correct someone over these kind of semantics.\nRuntime Environment Layers Runtime environments have their own runtime environments. If you download the Node binary for Linux, you’ll find that it’s just another ELF executable waiting to be run by the OS. So we could say—Javascript’s runtime environment is Node, and Node’s runtime environment is the operating system. Or you could also say—Javascript’s runtime environment is a combination of Node and the operating system. Again, dive deep enough and everyone’s runtime environment ends up being the operating system.\nInterpreting Languages Code does not always have to be compiled to OS/architecture-specific binaries. A common pattern is to execute your programs via interpreters. For example, a Python interpreter can read your Python source and produce corresponding machine instructions for your computer to execute. This conveniently makes Python source very portable. However, the Python interpreter itself is a compiled executable, built for a specific OS/architecture, who’s runtime environment is the operating system.\nThere is no such thing as interpreted languages or non-interpreted languages. A programming language is just a syntax. Python interpreters are the most popular way to execute Python source, but it’s not the only way. Programmers are creative. I’m sure developers have found ways to compile Python/Javascript into executables or have even created ways to interpret C.\nJava Program’s Runtime Environment What does it take to execute a Java program on your computer? This is very close to an executable’s runtime environment; all we need to do is to make sure our java source can turn into a proper bytecode.\nThis step is familiar to everyone. Java compiler takes your java code and create a properly formatted bytecode (.class) file, targeted for jvm.\nA subtler part of java compilers is that they also provide a collection of java runtime library that is automatically compiled into your program. The purpose of these libraries is to provide you—the programmer—with basic facilities to interact with the runtime environment. For a compiled java program as an bytecode executable in jvm, this environment is the jvm. For example, the java runtime library provides useful systems like JIT and java bytecode interpreter that allow you to manage memory and to help prepare it for execution, and many, many other things.\nAny library that allows a program to interact with the operating system will be heavily tailored to that operating system.Java by using JVM talks with operating system and executes the program in jvm and jvm facilitates all the necessary library and function calls. For java, the runtime libraries and functions we just discussed are part of a larger umbrella library referred to as JRE. This is an official standard.\nA Java program’s runtime environment is the jvm\u0026rsquo;s runtime environment, which in turn operating system. The important point to remember is that all code—from assembly to Javascript—needs some kind of environment in order execute, just like all recipes need some kind of kitchen to turn into food.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day45/","summary":"learning what is Runtime and Runtime environment and Java Runtime","title":"Day 45: learning about the internals of JVM and JRE (part 3)"},{"content":"\nJVM has 4 major parts\nClass Loader Runtime Data areas Execution engine Native interface and method libraries Classloader: Class loaders are responsible for loading Java classes dynamically to the JVM (Java Virtual Machine) during runtime. They\u0026rsquo;re also part of the JRE (Java Runtime Environment). So, the JVM doesn\u0026rsquo;t need to know about the underlying files or file systems in order to run Java programs thanks to class loaders.\nFurthermore, these Java classes aren\u0026rsquo;t loaded into memory all at once, but rather when they\u0026rsquo;re required by an application. This is where class loaders come into the picture. They\u0026rsquo;re responsible for loading classes into memory.\nclassloader subsystem does 3 things mainly\nClass Loading Linking Initializing Class Loading Mechanism: The Java platform uses a delegation model for loading classes. The basic idea is that every class loader has a parent class loader. When loading a class, a class loader first delegates the search for the class to its parent class loader before attempting to find the class itself.\nConstructors in java.lang.ClassLoader and its subclasses allow you to specify a parent when you instantiate a new class loader. If you don\u0026rsquo;t explicitly specify a parent, the virtual machine\u0026rsquo;s system class loader will be assigned as the default parent. The loadClass method in ClassLoader performs these tasks, in order, when called to load a class: If a class has already been loaded, it returns it. Otherwise, it delegates the search for the new class to the parent class loader. If the parent class loader does not find the class, loadClass calls the method findClass to find and load the class. The findClass method of ClassLoader searches for the class in the current class loader if the class wasn\u0026rsquo;t found by the parent class loader. You will probably want to override this method when you instantiate a class loader subclass in your application. The class java.net.URLClassLoader serves as the basic class loader for extensions and other JAR files, overriding the findClass method of java.lang.ClassLoader to search one or more specified URLs for classes and resources. 1 2 System.out.println(Day44.class.getClassLoader()); System.out.println(String.class.getClassLoader()); output:\njdk.internal.loader.ClassLoaders$AppClassLoader@512ddf17\rnull\rhere the class Day44 will be loaded by ApplicationClassLoader and String is loaded by BootStrapClassLoader A bootstrap or primordial class loader is the parent of all the others.Though, we can see that for the String, it displays null in the output. This is because the bootstrap class loader is written in native code, not Java, so it doesn\u0026rsquo;t show up as a Java class.\nLinking: After a class is located and its initial in-memory representation created in the JVM process, it is verified, prepared, resolved, and initialized.\nVerification\nmakes sure that the class is not corrupted and is structurally correct: its runtime constant pool is valid, the types of variables are correct, and the variables are initialized prior to being accessed. Verification can be turned off by supplying the -noverify option. If the JVM process does not run potentially malicious code, strict verification might not be required. Turning off the verification can speed up the startup of the JVM. Another benefit is that some classes, especially those generated on the fly by various tools, can be valid and safe for the JVM but unable to pass the strict verification process. In order to use such tools, the developer should disable this verification, which is often acceptable to do in a development environment.\nPreparation\nof a class involves initializing its static fields to the default values for their respective types. After preparation, fields of type int contain 0, references are null, and so forth.\nResolution\nof a class means checking that the symbolic references in the runtime constant pool actually point to valid classes of the required types. The resolution of a symbolic reference triggers loading of the referenced class. According to the JVM specification, this resolution process can be performed lazily, so it is deferred until the class is used.\nInitialization Initialization expects a prepared and verified class. It runs the class’s initializer. During initialization, the static fields are initialized to whatever values are specified in the code. The static initializer method that combines the code from all the static initialization blocks is also run. The initialization process should be run only once for every loaded class, so it is synchronized, especially because the initialization of the class can trigger the initialization of other classes and should be performed with care to avoid deadlocks.\nRuntime Data Areas: Runtime data areas consist of\nHeap Area Method Area Stack Area PC Registers Native Method Stack Heap Area: The Java Virtual Machine has a heap that is shared among all Java Virtual Machine threads. The heap is the run-time data area from which memory for all class instances and arrays is allocated. The heap is created on virtual machine start-up. Heap storage for objects is reclaimed by an automatic storage management system (known as a garbage collector); objects are never explicitly deallocated.\nHeap Memory can be accessed by any thread is further divided into three generations Young Generation, Old \u0026amp; PermGen(Permanent Generation). When object is created then it first go to Young generation(especially Eden space) when objects get old then it moves to Old/tenured Generation. In PermGen space all static \u0026amp; instance variables name-value pairs(name-references for object) are stored. Below is image showing heap structure of java.\nYou can manually increase heap size by some JVM parameters as shown in images. Suppose we have a simple class Day44 then increasing its memory by following parameters:-\nThere are types of heap memory spaces like eden , tenured and old spaces\njava -Xms=1M -XmX=2M Day44\rStack Area: Stack is generated with each thread created by program. It is associated by thread. Each Thread has its own stack. All local variables \u0026amp; function calls are stored in stack. It’s life depends upon Thread’s life as thread will be alive it will also and vice-versa. It can also be increased by manually:-\njava -Xss=512M Day44\rIt throws StackOverFlow error when stack get full.\nMethod Area: It is memory which is shared among all Threads like Heap. It is created on Java Virtual Machine startup. It contains the code actually a compiled code, methods and its data and fields. Runtime constant pool is also a part of Method Area. Runtime Constant pool is per class representation of constant Table. It contains all literals defined at compiled time and references which is going to be solved at runtime. Memory for it is by default allotted by JVM and can be increased if needed.\nNative Method Stack: An implementation of the Java Virtual Machine may use conventional stacks, also called \u0026ldquo;C stacks,\u0026rdquo; to support native methods (methods written in a language other than the Java programming language). Native method stacks may also be used by the implementation of an interpreter for the Java Virtual Machine\u0026rsquo;s instruction set in a language such as C. Java Virtual Machine implementations that cannot load native methods and that do not themselves rely on conventional stacks need not supply native method stacks. If supplied, native method stacks are typically allocated per thread when each thread is created.\nExecution Engine Execution Engine helps in executing the byte code by converting it into machine code and also interact with the memory area. So here are the components involved in executing:\nInterpreter:\nIt is responsible to read, interpret, and execute java program line by line. So if any method is called multiple times new interpretation required which is the main disadvantage of it.\nJIT Compiler (Just In Time):\nWhen we compile our Java program (e.g., using the javac command), we\u0026rsquo;ll end up with our source code compiled into the binary representation of our code – a JVM bytecode. This bytecode is simpler and more compact than our source code, but conventional processors in our computers cannot execute it.\nTo be able to run a Java program, the JVM interprets the bytecode. Since interpreters are usually a lot slower than native code executing on a real processor, the JVM can run another compiler which will now compile our bytecode into the machine code that can be run by the processor. This so-called just-in-time compiler is much more sophisticated than the javac compiler, and it runs complex optimizations to generate high-quality machine code\nThe JDK implementation by Oracle is based on the open-source OpenJDK project. This includes the HotSpot virtual machine, available since Java version 1.3. It contains two conventional JIT-compilers: the client compiler, also called C1 and the server compiler, called opto or C2.\nC1 is designed to run faster and produce less optimized code, while C2, on the other hand, takes a little more time to run but produces a better-optimized code. The client compiler is a better fit for desktop applications since we don\u0026rsquo;t want to have long pauses for the JIT-compilation. The server compiler is better for long-running server applications that can spend more time on the compilation.\nIntermediate Code Generator Code Optimizer Target Code Generator Profiler Note: Profiler is there to identify hotspot repeated methods inside JIT.\nIn order to provide native libraries to the Execution Engine, we have JNI that connects with Native libraries and the Execution Engine.\nIn the next part will discuss about what is runtime and java runtime environment\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day44/","summary":"learning about jvm  architecture","title":"Day 44: learning about the internals of JRE and JVM (Part 2)"},{"content":" First of all here inside out approach is followed to understand JVM then JRE and then JDK and along the path will try to explain Virtualization, Hypervisor, VM, various kind of VM, runtime, java program execution model and memory model.\nN.B JVM is a specification and it have many implementations.\nWhen we talk about JVM aka java virtual machine. So lets first understand what is virtual machine. To understand whats a virtual machine is we need to have some basic knowledge of key technologies like virtualization and hypervisor.\nVirtualization: Virtualization is the process of running a virtual instance of a computer system in a layer (Hypervisor) up from the actual hardware. Most commonly, it refers to running multiple operating systems on a computer system simultaneously. To the applications running on top of the virtualized machine, it can appear as if they are on their own dedicated machine, where the operating system, libraries, and other programs are unique to the guest virtualized system and unconnected to the host operating system which sits below it.\nHypervisor(VMM): A hypervisor aka VMM( Virtual Machine monitor) is a program for creating and running virtual machines. Hypervisors have traditionally been split into two classes: type one, or \u0026ldquo;bare metal\u0026rdquo; hypervisors that run guest virtual machines directly on a system\u0026rsquo;s hardware, essentially behaving as an operating system. Type two, or \u0026ldquo;hosted\u0026rdquo; hypervisors behave more like traditional applications that can be started and stopped like a normal program. In modern systems, this split is less prevalent, particularly with systems like KVM. KVM, short for kernel-based virtual machine, is a part of the Linux kernel that can run virtual machines directly, although you can still use a system running KVM virtual machines as a normal computer itself.\nVirtual Machine: A virtual machine is the emulated equivalent of a computer system that runs on top of another system. Virtual machines may have access to any number of resources:\ncomputing power, through hardware-assisted Limited access to the host machine\u0026rsquo;s CPU and memory one or more physical or virtual disk devices for storage A virtual or real network interface Other devices such as video cards, USB devices, or other hardware that are shared with the virtual machine. If the virtual machine is stored on a virtual disk, this is often referred to as a disk image. A disk image may contain the files for a virtual machine to boot, or, it can contain any other specific storage needs.\nThere are many type of VM and many layers where virtualization can be done and they are\nThere are five layers where virtualization can be possible. For understanding JVM (Java virtual Machine) that is a application level vm.\nApplication Level Virtualization: Virtualization at the application level virtualizes an application as a VM. On a OS, an application often runs as a process. Therefore, application-level virtualization is also known as process-level virtualization. The most popular approach is to deploy high level language (HLL)VMs.\nIn this scenario, the virtualization layer sits as an application program on top of the operating system, and the layer behaves like a VM that have the normal properties of an VM and can run programs written and compiled to a particular machine definition for that virtual machine. Any program written in the HLL and compiled for this VM will be able to run on it. The Microsoft .NET CLR and Java Virtual Machine (JVM) are two good examples of this class of VM.\nOther forms of application level virtualization are known as application isolation, application sandboxing, or application streaming. This process involves wrapping up the application in a layer that is isolated from the host OS and other applications. The result is an application that is much easier to distribute and remove from user workstations.\nSo now as JVM is an application that behaves like a virtual machine it has its own architecture.\nJVM: Java virtual machine (JVM) is a virtual machine that enables a computer to run Java programs as well as programs written in other languages that are also compiled to bytecode. The JVM is detailed by a specification that formally describes what is required in a JVM implementation. Having a specification ensures interoperability of Java programs across different implementations so that program authors using the Java Development Kit (JDK) need not worry about idiosyncrasies of the underlying hardware platform.\nJVM has some important tasks to perform and those are\nLoading Code Verifying Code Executing Code Providing Runtime Environment JVM Architecuture\nIn Part 2 details discussion about JVM will be posted\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day43/","summary":"Understating the internals of jre and jvm how does they work to run a java program","title":"Day 43: learning about the internals of jre, jvm (Part 1)"},{"content":"Annotations are type of metadata ( data that provides information about other data) provide data about a program that is not part of the program itself. They don\u0026rsquo;t have any effect on operations of the program.\nimplemented two custom annotation one for Class level named ClassLevelAnnotation and another one is Field level named FieldLevelAnnotation\n1 2 3 4 5 6 7 8 9 @Retention(RetentionPolicy.RUNTIME) @Target(ElementType.TYPE) @interface ClassLevelAnnotation { } @Retention(RetentionPolicy.RUNTIME) @Target(ElementType.FIELD) @interface FieldLevelAnnotation { } Retention: It declares the availability of the annotation. if its given RetentionPolicy.RUNTIME that means it will be available in the runtime. If no retention policy is there then it takes the default retention policy which is RetentionPolicy.CLASS that means the annotation will be stored in the class file but not be available at the runtime.\nTarget: It gives the option to declare which java elements the custom annotation can be used to annotate. ElementType.TYPE means it targets a class type and can be used to annotate class. ElementType.FIELD means it targets a class\u0026rsquo;s field and can be used to annotate it.\nHere in this code there is another type of annotation that is ElementType.ANNOTATION_TYPE which is used to annotate another annotation. For example in the implementation the @Target and @Retention\nNow the annotations are placed on a class\n1 2 3 4 5 @ClassLevelAnnotation static class AnotherClass { @FieldLevelAnnotation public int firstNumber; } now in the main class there are two methods implemented to check if the class is annotated with a class level annotation and a field is annotated with a field level annotation.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 class Day42 { public static void main(String[] args) throws NoSuchFieldException { System.out.println(isAnnotatedClass(AnotherClass.class)); System.out.println(isAnnotatedField(AnotherClass.class,\u0026#34;firstNumber\u0026#34;)); } } private static boolean isAnnotatedField(Class\u0026lt;?\u0026gt; clasz, String fieldName) throws NoSuchFieldException { return clasz.getDeclaredField(fieldName).isAnnotationPresent(FieldLevelAnnotation.class); } private static boolean isAnnotatedClass(Class\u0026lt;?\u0026gt; clasz) { return clasz.isAnnotationPresent(ClassLevelAnnotation.class); } in isAnnotatedField() it takes a class and the fieldName that needs to be checked if it had annotation on it. isAnnotatedClass() takes a class and checks if any annotations are present on that class using the java.lang.Class\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day42/","summary":"implementing custom annotations and check if its a class level or field level annotation.","title":"Day 42: using custom annotations and check if its a class level or field level annotation"},{"content":"Generating unique id\u0026rsquo;s based on device MAC address and default timezone epoch time of given or a default time. These id\u0026rsquo;s can be generated in multiple machines as device mac address is used to create the id.\n64 bit Id format is\nEpoch Time - (41 bits) is used of epoch time and added to the id creation process. Max timestamp that can be represented is 2^41\nMachine Id - (10 bits) machine id is calculated from MAC address and as 10 bits considered so max machines allowed will be 2^10\nLocal counter per machine: sequence bits (12 bits) are a local counter for each machines. Max value would be 2 ^ 12.\nThe one remaining sign bit is remained and its set to 0\nCustomUIDSupplier class is a Supplier\u0026lt;Long\u0026gt; that provides the long id\u0026rsquo;s.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 class CustomUIDSupplier implements Supplier\u0026lt;Long\u0026gt; { private static final int MACHINE_MAC_ID_BITS = 10; private static final int SEQUENCE_BITS = 12; private static final int maxMachineID = (int)(Math.pow(2, MACHINE_MAC_ID_BITS) - 1); private static final int maxSequence = (int)(Math.pow(2, SEQUENCE_BITS) - 1); private static final long CUSTOM_EPOCH = LocalDateTime .parse(\u0026#34;2012-01-01T00:00:00\u0026#34;, DateTimeFormatter.ISO_LOCAL_DATE_TIME) .atZone(ZoneId.systemDefault()) .toInstant() .toEpochMilli(); private final int machineMacId; private volatile long lastTimestamp = -1L; private volatile long sequence = 0L; public CustomUIDSupplier(int machineMacId) { if(machineMacId \u0026lt; 0 || machineMacId \u0026gt; maxMachineID) { throw new IllegalArgumentException(String.format(\u0026#34;NodeId must be between %d and %d\u0026#34;, 0, maxMachineID)); } this.machineMacId = machineMacId; } public CustomUIDSupplier() { this.machineMacId = createNodeId(); } // Get current timestamp in milliseconds, adjust for the custom epoch. private static long timestamp(long customEpoch) { long epochInThisMoment = Instant.now().toEpochMilli(); if (customEpoch != 0) return epochInThisMoment - customEpoch; else return epochInThisMoment; } // Block and wait till next millisecond private long waitNextMillis(long currentTimestamp) { while (currentTimestamp == lastTimestamp) { currentTimestamp = timestamp(CUSTOM_EPOCH); } return currentTimestamp; } private int createNodeId() { int machineMacId; try { StringBuilder sb = new StringBuilder(); Enumeration\u0026lt;NetworkInterface\u0026gt; networkInterfaces = NetworkInterface.getNetworkInterfaces(); while (networkInterfaces.hasMoreElements()) { NetworkInterface networkInterface = networkInterfaces.nextElement(); byte[] mac = networkInterface.getHardwareAddress(); if (mac != null) { for(int i = 0; i \u0026lt; mac.length; i++) { sb.append(String.format(\u0026#34;%02X\u0026#34;, mac[i])); } } } machineMacId = sb.toString().hashCode(); } catch (Exception ex) { machineMacId = (new SecureRandom().nextInt()); } machineMacId = machineMacId \u0026amp; maxMachineID; return machineMacId; } @Override public Long get() { return nextId(); } public synchronized long nextId() { long currentTimestamp = timestamp(CUSTOM_EPOCH); if(currentTimestamp \u0026lt; lastTimestamp) { throw new IllegalStateException(\u0026#34;Invalid TimeStamp!\u0026#34;); } if (currentTimestamp == lastTimestamp) { sequence = (sequence + 1) \u0026amp; maxSequence; if(sequence == 0) { // Sequence Exhausted, wait till next millisecond. currentTimestamp = waitNextMillis(currentTimestamp); } } else { // reset sequence to start with zero for the next millisecond sequence = 0; } lastTimestamp = currentTimestamp; long id = currentTimestamp \u0026lt;\u0026lt; (MACHINE_MAC_ID_BITS + SEQUENCE_BITS); id |= ((long) machineMacId \u0026lt;\u0026lt; SEQUENCE_BITS); id |= sequence; return id; } } In CustomerUIDSupplier class custom epoch is passed or calculated along with machine id that can also be passed in the constructor. SecureRandom is used in case the machine id can not be calculated.\nnextId() which in turn is called inside get() of the supplier is calculated based on the sequence and timestamps and machine\u0026rsquo;s mac based id\n1 2 3 4 5 6 7 class Day41 { public static void main(String[] args) { CustomUIDSupplier customUIDSupplier = new CustomUIDSupplier(); Stream\u0026lt;Long\u0026gt; longStream = Stream.generate(customUIDSupplier); longStream.limit(10).forEach(System.out::println); } } in the main() a stream is generated from supplier that will in turn provide the unique id\u0026rsquo;s and then 10 of the id\u0026rsquo;s are taken and printed\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day41/","summary":"Generating unique id\u0026rsquo;s based on device MAC address and epoch time","title":"Day 41: Generating unique id's based on device MAC address and epoch time"},{"content":"In one of my previous post i tired to make python like generator in java using Supplier function and IntStream. That didnt fully behaved like a generator. So tried implementing this time using custom Iterator for lazy evaluation of the items in the collections and using a Thread to produce the items that will be yield when calling the generator.\nin this repl you can execute the generator named InfiniteGenerator which extended an abstract class Generator\u0026lt;T\u0026gt; which in turn implements Iterable\u0026lt;T\u0026gt; to implement a custom iterator of a given type. Any class implementing Generator\u0026lt;T\u0026gt; class will be able to act like an generator if a given implementation that provides with items.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day40/","summary":"A python like generator implementation using custom Iterator, Thread and ThreadGroup","title":"Day 40: Generator implementation using custom Iterator and Threads. "},{"content":"I faced some issues while working with LocalDateTime and converting localdatetime to UTC and vice versa. Thats why created a method that takes LocalDateTime Object and converts it to UTC date time or UTC date time to local date time of the timzone thats the JVM is in.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 public static ZoneOffset getTimeZoneOffset(LocalDateTime dateTime) { TimeZone zone = TimeZone.getDefault(); // translate DayOfWeek values to Calendar\u0026#39;s int dayOfWeek; if (dateTime.getDayOfWeek().getValue() == 7) { dayOfWeek = 1; } else { dayOfWeek = dateTime.getDayOfWeek().getValue() + 1; } // get the offset used in the timezone, at the specified date // int offset = zone.getOffset(1, now.getYear(), now.getMonthValue() - 1, // now.getDayOfMonth(), dayOfWeek, now.getNano() / 1000000); int offset = 0; if (zone.useDaylightTime()) { offset = zone.getDSTSavings(); } offset = zone.getRawOffset(); return ZoneOffset.ofTotalSeconds(offset / 1000); } in this getTimeZoneOffset() using the LocalDateTime and TimeZone.getDefault() which in turns will get the default timezone from the jvm instance. zone.getRawOffset() of TimeZone class will return the raw offset.\nRaw Offset: Returns the amount of time in milliseconds to add to UTC to get standard time in this time zone. Because this value is not affected by daylight saving time, it is called raw offset.\nDST Savings: Returns the amount of time to be added to local standard time to get local wall clock time. The default implementation returns 3600000 milliseconds (i.e., one hour) if a call to useDaylightTime() returns true. Otherwise, 0 (zero) is returned.\nThen using the getTimeZoneOffset() to calculate the local to UTC also UTC to local time.\n1 2 3 4 5 6 7 8 9 10 11 12 public static LocalDateTime localToUTCViceVersa( LocalDateTime localDatTime, ZoneOffset offset, boolean switcher) { return localDatTime // convert to timezone\u0026#39;s offset (local -\u0026gt; UTC) // or convet to UTC (UTC -\u0026gt; local) .atOffset(switcher ? offset : ZoneOffset.UTC) // convert to UTC (local -\u0026gt; UTC) // or convert to TimeZone offset (UTC -\u0026gt; local) .withOffsetSameInstant(switcher ? ZoneOffset.UTC : offset) // get LocalDateTime .toLocalDateTime(); } withOffsetSameInstant() : Returns a copy of this OffsetDateTime with the specified offset ensuring that the result is at the same instant. This method returns an object with the specified ZoneOffset and a LocalDateTime adjusted by the difference between the two offsets. This will result in the old and new objects representing the same instant. This is useful for finding the local time in a different offset. For example, if this time represents 2007-12-03T10:30+02:00 and the offset specified is +03:00, then this method will return 2007-12-03T11:30+03:00\nin localToUTCViceVersa() which takes a LocalDateTime and ZoneOffset and a switcher to switch to local -\u0026gt; UTC or UTC -\u0026gt; local. which takes the offset and makes the OffsetDateTime and then using the withOffsetSameInstant() we get local date time in the local time zone or in UTC.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day39/","summary":"Converting LocalDateTime to UTC and vice versa with respect to timezone and Day Light saving time.","title":"Day 39: Local date time to UTC date time and vice versa with respect to timezone and Day light saving"},{"content":"Feature toggle is used to easily enable and disable features for some group of users of the applications. Here Usergroup.java is used to define the user groups which are free and paid users. TieredFeatureToggle.java uses this paid user information to show custom messsage to paid users.\nHere is the example.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day37/","summary":"Feature toggle service to enable disable any feature for paid and unpaid users","title":"Day 37: Feature toggle to enable disable features for paid and unpaid user"},{"content":"Checked Exceptions: Exceptions that are checked during the compile time are called the checked exception. Basically compiler check for the exception to verify that a method or class that is throwing the exception have code that have given the code contingency. A way of handling with try-catch.\n1 2 File file = new File(\u0026#34;/path/of/a/non/existing/file); FileInputStream fileInputStream = new FileInputStream(file); For example, here when instantiating File object if the file is not present then it throws FileNotFoundException and its super class is Exception class which is checked at compile time.\nUnchecked Exceptions: Exceptions that are not checked during compile time are called unchecked exceptions. These exceptions occurs in run time.Furthermore, we don\u0026rsquo;t have to declare unchecked exceptions in a method with the throws keyword. Some common unchecked exceptions in Java are NullPointerException, ArrayIndexOutOfBoundsException and IllegalArgumentException.\n1 2 3 4 5 private static void divideByZero() { int numerator = 1; int denominator = 0; int result = numerator / denominator; } For example, here when result is calculated it casues ArithmaticException which is a runtime exception.\nWhen to use Checked and Unchecked: From Oracle Java Documentation provides guidance on when to use checked exceptions and unchecked exceptions:\n“If a client can reasonably be expected to recover from an exception, make it a checked exception. If a client cannot do anything to recover from the exception, make it an unchecked exception.”\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day38/","summary":"One of the case where to use checked and another for unchecked exceptions","title":"Day 38: One of the cases to use check and unchecked exceptions"},{"content":"Last week CVE (Common Vulnerabilities and Exposures) -2021-44228 in Apache Log4J logging library which is a RCE (Remote Code Execution) class vulnerability was reported.\nLog4J is a popular logging library which is open sourced and used by many projects all over the tech space.\nPatch and resolvable techniques are already provided in many blogs and websites.\nRecently i learned about from banned dependencies from @gunnarmorling\u0026rsquo;s tweet . Maven\u0026rsquo;s enforcer plugin can ban certain dependencies using rules.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 \u0026lt;plugin\u0026gt; \u0026lt;groupId\u0026gt;org.apache.maven.plugins\u0026lt;/groupId\u0026gt; \u0026lt;artifactId\u0026gt;maven-enforcer-plugin\u0026lt;/artifactId\u0026gt; \u0026lt;version\u0026gt;3.0.0\u0026lt;/version\u0026gt; \u0026lt;executions\u0026gt; \u0026lt;execution\u0026gt; \u0026lt;id\u0026gt;ban-vulnerable-log4j2-versions\u0026lt;/id\u0026gt; \u0026lt;phase\u0026gt;validate\u0026lt;/phase\u0026gt; \u0026lt;goals\u0026gt; \u0026lt;goal\u0026gt;enforce\u0026lt;/goal\u0026gt; \u0026lt;/goals\u0026gt; \u0026lt;configuration\u0026gt; \u0026lt;rules\u0026gt; \u0026lt;bannedDependencies\u0026gt; \u0026lt;excludes\u0026gt; \u0026lt;exclude\u0026gt;org.apache.logging.log4j:log4j-core:(,2.15.0)\u0026lt;/exclude\u0026gt; \u0026lt;/excludes\u0026gt; \u0026lt;message\u0026gt;Maven error because -\u0026gt; log4j is vulnerable to remote code execution\u0026lt;/message\u0026gt; \u0026lt;/bannedDependencies\u0026gt; \u0026lt;/rules\u0026gt; \u0026lt;fail\u0026gt;true\u0026lt;/fail\u0026gt; \u0026lt;/configuration\u0026gt; \u0026lt;/execution\u0026gt; \u0026lt;/executions\u0026gt; \u0026lt;/plugin\u0026gt; here \u0026lt;bannedDependencies\u0026gt; rule enforces to exclude any log4j dependency to get excluded if its not 2.15.0 or greater. searchTransitive is by default true so it will check transitive dependencies and apply this rule for them.\nOn gradle there are constraints that can help to do similar rule enforcement for excluding vulnerable dependencies from project. That i learned from @CedricChampeau tweet.\n1 2 3 4 5 6 7 8 9 10 11 dependencies { constraints { implementations (\u0026#34;org.apache.logging.log4j:log4j-core\u0026#34;) { version { strictly(\u0026#34;[2.15,3[\u0026#34;) prefer(\u0026#34;2.15.0\u0026#34;) } because(\u0026#34;Maven error because -\u0026gt; log4j is vulnerable to remote code execution) } } } ","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day36/","summary":"Many 3rd party libraries can have vulnerable dependency like recently reported Log4j 2. so to ban them from getting resolved in your maven or gradle project you can use maven-enforcer plugin.","title":"Day 36: Learned about banned dependencies and how to ban transitive dependency in a maven or gradle project"},{"content":"While working need a structure like Pair in java so tired to emulate the behavior using AbstractMap.SimpleEntry\u0026lt;\u0026gt;() and a technique shared by @starbuxman in his tweet about creating tuple using new Object(){} implementation.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 class Day35 { public static void main(String[] args) { DoubleStream doubleStream = DoubleStream.iterate(1.0 , d -\u0026gt; d+0.5); List\u0026lt;AbstractMap.SimpleEntry\u0026lt;Double, Double\u0026gt;\u0026gt; pairs = doubleStream .limit(10) .mapToObj(number -\u0026gt; new AbstractMap.SimpleEntry\u0026lt;\u0026gt;(new Random().nextDouble(), number)) .collect(Collectors.toList()); pairs.forEach(pair -\u0026gt; { System.out.println(pair.getKey()); System.out.println(pair.getValue()); }); Stream.generate(() -\u0026gt; new Random().nextDouble()) .limit(10) .map(lon -\u0026gt; new Object() { double lattitude = new Random().nextDouble(); double longitude = lon; }).forEach(tuple -\u0026gt; { System.out.println(tuple.lattitude); System.out.println(tuple.longitude); }); } } ","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day35/","summary":"Pair behavior in java","title":"Day 35: Emulating Pair in java using AbstractMap.SimpleEntry\u003c\u003e() and new Object() {} implementation"},{"content":"Problem: https://adventofcode.com/2021/day/2\nPlease for changing the code fork it in repl.it or copy\nJava Solution Rust Solution Golang Solution ","permalink":"https://mohibul.pages.dev/posts/adeventofcode2021/day02/","summary":"Listed here are the solutions of AOC2021 Day02 in Java, Rust, Go","title":"Advent of Code Day 02"},{"content":"Problem: https://adventofcode.com/2021/day/1\nPlease for changing the code fork it in repl.it or copy\nJava Solution Rust Solution Golang Solution ","permalink":"https://mohibul.pages.dev/posts/adeventofcode2021/day01/","summary":"Listed here are the solutions of AOC2021 Day01 in Java, Rust, Go","title":"Advent of Code Day 01"},{"content":"Inspecting JVM runtime memory and available processor count with Runtime.getRuntime().\nTranscribed from the original LinkedIn image post.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 import org.apache.commons.io.FileUtils; class Day34 { public static void main(String[] args) { int availableCoreCountToJVM = Runtime.getRuntime().availableProcessors(); long maxMemoryUsableToJVM = Runtime.getRuntime().maxMemory(); long availableFreeMemoryInJVM = Runtime.getRuntime().freeMemory(); long JVMTotalMemory = Runtime.getRuntime().totalMemory(); System.out.println(\u0026#34;Amount of max memory for JVM : \u0026#34; + FileUtils.byteCountToDisplaySize(maxMemoryUsableToJVM)); System.out.println(\u0026#34;Amount of total memory in JVM : \u0026#34; + FileUtils.byteCountToDisplaySize(JVMTotalMemory)); System.out.println(\u0026#34;Amount of free memory in JVM : \u0026#34; + FileUtils.byteCountToDisplaySize(availableFreeMemoryInJVM)); System.out.println(\u0026#34;Available CPU core count in JVM: \u0026#34; + availableCoreCountToJVM); } } The original LinkedIn graphic is preserved below.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day34/","summary":"Inspecting JVM runtime memory and available processor count with Runtime.getRuntime().","title":"Day 34: Inspecting JVM runtime memory and processor count with Runtime"},{"content":"Getting a programmer joke asynchronously with CompletableFuture, HttpClient, and Jackson.\nTranscribed from the original LinkedIn image post.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 import com.fasterxml.jackson.annotation.JsonIgnoreProperties; import com.fasterxml.jackson.annotation.JsonProperty; import com.fasterxml.jackson.core.type.TypeReference; import com.fasterxml.jackson.databind.ObjectMapper; import java.io.IOException; import java.net.URI; import java.net.URISyntaxException; import java.net.http.HttpClient; import java.net.http.HttpRequest; import java.net.http.HttpResponse; import java.util.concurrent.CompletableFuture; import java.util.concurrent.ExecutionException; import java.util.concurrent.TimeUnit; import java.util.logging.Logger; class Day33 { private static final Logger logger = Logger.getLogger(Day33.class.getName()); public static void main(String[] args) { try { String webSite = CompletableFuture.supplyAsync(Day33::getWebSite) .completeOnTimeout(\u0026#34;Emptiness itself is a joke\u0026#34;, 5, TimeUnit.SECONDS) .get(); System.out.println(\u0026#34;Joke of the day: \\n\u0026#34;); System.out.println(webSite); } catch (ExecutionException | InterruptedException e) { logger.severe(\u0026#34;Couldnt execute or got interrupted cause \u0026#34; + e); } } private static String getWebSite() { try { HttpClient httpClient = HttpClient.newHttpClient(); HttpRequest httpRequest = HttpRequest .newBuilder() .uri(new URI(\u0026#34;https://v2.jokeapi.dev/joke/Programming?blacklistFlags=nsfw,religious,political,racist,sexist,explicit\u0026amp;type=single\u0026#34;)) .GET() .header(\u0026#34;Accept\u0026#34;, \u0026#34;application/json\u0026#34;) .build(); HttpResponse\u0026lt;String\u0026gt; response = httpClient.send( httpRequest, HttpResponse.BodyHandlers.ofString() ); String joke = response.body(); ObjectMapper mapper = new ObjectMapper(); Joke jokeObj = mapper.readValue(joke, new TypeReference\u0026lt;Joke\u0026gt;() {}); return jokeObj.joke; } catch (InterruptedException | URISyntaxException | IOException e) { throw new RuntimeException(e); } } @JsonIgnoreProperties(ignoreUnknown = true) static class Joke { @JsonProperty(\u0026#34;joke\u0026#34;) public String joke; } } The original LinkedIn graphic is preserved below.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day33/","summary":"Getting a programmer joke asynchronously with CompletableFuture, HttpClient, and Jackson.","title":"Day 33: Getting a programmer joke asynchronously with CompletableFuture, HttpClient, and Jackson"},{"content":"Using Objects.equals() when one of the values may be null.\nTranscribed from the original LinkedIn image post.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 import java.util.Objects; class Day32 { public static void main(String[] args) { String file1Name = \u0026#34;data.zip\u0026#34;; String file2Name = \u0026#34;data.json\u0026#34;; System.out.println(Objects.equals(true, file2Name)); System.out.println(file2Name.equals(file1Name)); // Changing file2Name to data.zip to get equality. file2Name = \u0026#34;data.zip\u0026#34;; System.out.println(Objects.equals(file1Name, file2Name)); file2Name = null; System.out.println(Objects.equals(file1Name, file2Name)); // This will give NullPointerException as null.equals() is not possible. System.out.println(file2Name.equals(file1Name)); } } The original LinkedIn graphic is preserved below.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day32/","summary":"Using Objects.equals() when one of the values may be null.","title":"Day 32: Using Objects.equals() when values may be null"},{"content":"Compressing and decompressing text with Deflater and Inflater.\nTranscribed from the original LinkedIn image post. The original graphic used a longer novel excerpt as sample text; this version keeps the same compression flow with a neutral sample paragraph.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 import java.nio.charset.StandardCharsets; import java.util.zip.DataFormatException; import java.util.zip.Deflater; import java.util.zip.Inflater; class Day31 { public static void main(String[] args) { String message = \u0026#34;A sample paragraph that will be compressed and decompressed to show how Deflater and Inflater work in Java.\u0026#34;; CompressedData compressedData = compressString(message); System.out.println(\u0026#34;Original String length = \u0026#34; + message.length()); String compressedString = new String(compressedData.compressedData, StandardCharsets.UTF_8); System.out.println(\u0026#34;Compressed string length = \u0026#34; + compressedString.length()); String deCompressedString = decompressString(compressedData); System.out.println(\u0026#34;Compressed String:\\n\u0026#34; + compressedString); System.out.println(\u0026#34;Decompressed string:\\n\u0026#34; + deCompressedString); } static class CompressedData { public byte[] compressedData; public int compressedSize; public CompressedData(byte[] compressedData, int compressedSize) { this.compressedData = compressedData; this.compressedSize = compressedSize; } } static CompressedData compressString(String contentPath) { Deflater deflater = new Deflater(); byte[] content = contentPath.getBytes(StandardCharsets.UTF_8); deflater.setInput(content); deflater.finish(); byte[] compressed = new byte[content.length]; int compressedSize = deflater.deflate(compressed, 0, contentPath.length(), Deflater.NO_FLUSH); deflater.end(); return new CompressedData(compressed, compressedSize); } static String decompressString(CompressedData compressedString) { Inflater inflater = new Inflater(); inflater.setInput(compressedString.compressedData, 0, compressedString.compressedSize); byte[] original = new byte[compressedString.compressedData.length]; int originalLength = 0; try { originalLength = inflater.inflate(original); } catch (DataFormatException e) { e.printStackTrace(); } inflater.end(); return new String(original, 0, originalLength, StandardCharsets.UTF_8); } } The original LinkedIn graphic is preserved below.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day31/","summary":"Compressing and decompressing text with Deflater and Inflater.","title":"Day 31: Compressing and decompressing text with Deflater and Inflater"},{"content":"Exploring a generator-style approach in Java with Supplier and IntStream.generate().\nTranscribed from the original LinkedIn image post.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 import java.util.function.Function; import java.util.function.IntSupplier; import java.util.stream.IntStream; class Day30 { public static void main(String[] args) { Function\u0026lt;Integer, Integer\u0026gt; squareFunction = integer -\u0026gt; integer * integer; SupplierProvider squareSupplier = new SupplierProvider(squareFunction); IntStream myStream = IntStream.generate(squareSupplier); myStream.limit(10).forEach(System.out::println); System.out.println(\u0026#34;---------------- INTERVAL ----------------\u0026#34;); IntStream stream = IntStream.generate(squareSupplier); stream.limit(5).forEach(System.out::println); } static class SupplierProvider implements IntSupplier { int i = 0; Function\u0026lt;Integer, Integer\u0026gt; function; public SupplierProvider(Function\u0026lt;Integer, Integer\u0026gt; function) { this.function = function; } @Override public int getAsInt() { i++; return function.apply(i); } } } The original LinkedIn graphic is preserved below.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day30/","summary":"Exploring a generator-style approach in Java with Supplier and IntStream.generate().","title":"Day 30: Exploring a generator-style approach in Java with Supplier and IntStream"},{"content":"Using ParameterizedTest and MethodSource to run the same file-presence test with different inputs.\nTranscribed from the original LinkedIn image post.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 import static org.junit.jupiter.api.Assertions.assertEquals; import java.io.IOException; import java.nio.file.Files; import java.nio.file.Path; import java.nio.file.Paths; import java.util.ArrayList; import java.util.List; import java.util.stream.Collectors; import java.util.stream.Stream; import org.junit.jupiter.params.ParameterizedTest; import org.junit.jupiter.params.provider.Arguments; import org.junit.jupiter.params.provider.MethodSource; class Day29 { final static String PATH = \u0026#34;/home/mohibulhasan/Downloads\u0026#34;; public boolean isFileAvailable(String relativeFilePath) { List\u0026lt;String\u0026gt; fileNames = new ArrayList\u0026lt;\u0026gt;(); try (Stream\u0026lt;Path\u0026gt; paths = Files.walk(Paths.get(PATH))) { fileNames = paths .filter(Files::isRegularFile) .map(Path::toAbsolutePath) .map(Path::toString) .collect(Collectors.toList()); } catch (IOException e) { e.printStackTrace(); } return fileNames.contains(relativeFilePath); } @ParameterizedTest(name = \u0026#34;{index} - Test with fileName : {0}\u0026#34;) @MethodSource(\u0026#34;fileNameProvider\u0026#34;) public void testFilePresentOrNot(String fileName, boolean expectedResult) { assertEquals(expectedResult, isFileAvailable(fileName)); } public static Stream\u0026lt;Arguments\u0026gt; fileNameProvider() { return Stream.of( Arguments.of(PATH + \u0026#34;/data.json\u0026#34;, true), Arguments.of(PATH + \u0026#34;/404.txt\u0026#34;, false) ); } } The original LinkedIn graphic is preserved below.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day29/","summary":"Using ParameterizedTest and MethodSource to run the same file-presence test with different inputs.","title":"Day 29: ParameterizedTest and MethodSource for repeated test inputs"},{"content":"Arrays.asList() and new ArrayList\u0026lt;\u0026gt;() both return lists, but they do not behave the same way.\nTranscribed from the original LinkedIn image post.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 import java.util.ArrayList; import java.util.Arrays; import java.util.List; class Day28 { public static void main(String[] args) { List\u0026lt;String\u0026gt; filePathsFromList = new ArrayList\u0026lt;\u0026gt;(); filePathsFromList.add(\u0026#34;/home/mohibulhasan/Downloads\u0026#34;); filePathsFromList.add(\u0026#34;/home/mohibulhasan/Pictures\u0026#34;); String[] paths = new String[]{ \u0026#34;/home/mohibulhasan/Downloads\u0026#34;, \u0026#34;/home/mohibulhasan/Pictures\u0026#34; }; List\u0026lt;String\u0026gt; filePathsFormArray = Arrays.asList(paths); // Calling add on a list created from Arrays.asList() // will throw UnsupportedOperationException. filePathsFormArray.add(\u0026#34;/home/mohibulhasan/Applications\u0026#34;); filePathsFromList.add(\u0026#34;/home/mohibulhasan/Applications\u0026#34;); System.out.println(\u0026#34;Arrays.asList() returns ArrayList? \u0026#34; + (filePathsFormArray instanceof ArrayList)); System.out.println(\u0026#34;new ArrayList\u0026lt;\u0026gt;() returns ArrayList? \u0026#34; + (filePathsFromList instanceof ArrayList)); } } The original LinkedIn graphic is preserved below.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day28/","summary":"Arrays.asList() and new ArrayList\u0026lt;\u0026gt;() both return lists, but they do not behave the same way.","title":"Day 28: Arrays.asList() and new ArrayList\u003c\u003e() are not the same"},{"content":"Implementing a Unix pipe-like class in Java with Function composition.\nTranscribed from the original LinkedIn image post.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 import java.time.LocalDateTime; import java.time.format.DateTimeFormatter; import java.util.ArrayList; import java.util.List; import java.util.function.Function; import java.util.regex.Matcher; import java.util.regex.Pattern; class Day27 { public static void main(String[] args) { Function\u0026lt;String, String\u0026gt; awk1 = commandLine -\u0026gt; { final String regex = \u0026#34;(\\\\d{4}-\\\\d{2}-\\\\d{2}\\\\s\\\\d{2}:\\\\d{2}:\\\\d{2})\u0026#34;; final Pattern pattern = Pattern.compile(regex, Pattern.MULTILINE); final Matcher matcher = pattern.matcher(commandLine); if (matcher.find()) { return matcher.group(0); } return \u0026#34;\u0026#34;; }; Function\u0026lt;List\u0026lt;String\u0026gt;, String\u0026gt; grepFirstOccurrence = history -\u0026gt; history.stream() .filter(str -\u0026gt; str.contains(\u0026#34;docker\u0026#34;)) .findFirst() .orElse(\u0026#34;\u0026#34;); Function\u0026lt;String, LocalDateTime\u0026gt; dateTimeToLocalDateTime = dateTimeText -\u0026gt; { DateTimeFormatter dateTimeFormatter = DateTimeFormatter.ofPattern(\u0026#34;yyyy-MM-dd HH:mm:ss\u0026#34;); return LocalDateTime.parse(dateTimeText, dateTimeFormatter); }; List\u0026lt;String\u0026gt; history = new ArrayList\u0026lt;\u0026gt;(); history.add(\u0026#34;876 2021-10-12 15:59:47 docker ps\u0026#34;); history.add(\u0026#34;877 2021-10-12 13:33:18 docker volume ls\u0026#34;); LocalDateTime filteredResult = new Pipeline\u0026lt;\u0026gt;(grepFirstOccurrence) .pipeableCommand(awk1) .pipeableCommand(dateTimeToLocalDateTime) .execute(history); System.out.println(filteredResult); } static class Pipeline\u0026lt;input, output\u0026gt; { private final Function\u0026lt;input, output\u0026gt; function; Pipeline(Function\u0026lt;input, output\u0026gt; function) { this.function = function; } \u0026lt;intermediate\u0026gt; Pipeline\u0026lt;input, intermediate\u0026gt; pipeableCommand(Function\u0026lt;output, intermediate\u0026gt; anotherFunction) { return new Pipeline\u0026lt;\u0026gt;(input -\u0026gt; anotherFunction.apply(function.apply(input))); } output execute(input input) { return function.apply(input); } } } The original LinkedIn graphic is preserved below.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day27/","summary":"Implementing a Unix pipe-like class in Java with Function composition.","title":"Day 27: Implementing a Unix pipe-like class in Java with Function composition"},{"content":"Writing a method which can take another method and run that and if needed retry that execution given number of times. Using Runnable and scheduleAtFixedRate() from timer class.\nTimerTask along with Timer class can schedule tasks using background thread. Timer has two type of scheduling option * schedule (Fixed delayed execution) * scheduleAtFixedRate (Fixed rate execution)\nFixed Delay Execution: Execution after the first execution does not depend on the start time of the execution rather than the completion of the first execution then it starts the second execution\nFixed Rate Execution: Execution after the first execution does depend on the start time of the execution. Second and execution afterwards\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 class Day26 { public static void main(String[] args) { retryMethod(() -\u0026gt; uptimeCheck(\u0026#34;https://twitter.com\u0026#34;), 3, 5000L); } public static void retryMethod(Runnable function, int retryCount, long retryInterval) { Timer timer = new Timer(); timer.scheduleAtFixedRate(new TimerTask() { int count = 0; @Override public void run() { count++; if (count == retryCount) { timer.cancel(); } try { function.run(); } catch (Exception e) { e.printStackTrace(); } } }, 0, retryInterval); } public static void checkAvailability(String url) { HttpClient httpClient = HttpClient.newHttpClient(); int statusCode = 0; HttpRequest httpRequest = HttpRequest .newBuilder() .uri(URI.create(url)) .GET() .build(); try { statusCode = httpClient.send(httpRequest, HttpResponse.BodyHandlers.ofString()).statusCode(); } catch (IOException | InterruptedException e) { e.printStackTrace(); } if (statusCode == 200) System.out.println(url + \u0026#34; is up and status: \u0026#34; + statusCode); else System.out.println(url + \u0026#34; is down and status: \u0026#34; + statusCode); } } Here retryMethod is a method that takes a runnable and interval time and retry count to run the runnable using TimerTask and Timer\u0026rsquo;s scheduleAtFixedRate.\nFurthermore here checkAvailability method takes an url and check if the url returns 200 thus getting a partial knowledge about if the site is up or down.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day26/","summary":"Writing a method to retry another method execution, using TimerTask\u0026rsquo;s schedule() and scheduleAtFixedRate to check availability","title":"Day 26: Writing a method to retry another method execution"},{"content":"Using a implementation of supplier creating and caching a large file (any Class in this case that will initialize a very large object) lazily and thread-safe.\nTranscribed from the original LinkedIn image post.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 import java.io.File; import java.io.FileNotFoundException; import java.util.Optional; import java.util.function.Supplier; class Day25 { public static void main(String[] args) throws Exception { FileLoader largeFileLoader = new FileLoader(); File largeFile = largeFileLoader .getLargeFile() .orElseThrow(FileNotFoundException::new); System.out.println(largeFile.getName()); } static class FileLoader { private Supplier\u0026lt;Optional\u0026lt;File\u0026gt;\u0026gt; largeFile = () -\u0026gt; createAndCacheFile(\u0026#34;/home/mohibulhasan/Downloads/data.json\u0026#34;); public FileLoader() { System.out.println(\u0026#34;File Loader created\u0026#34;); } public Optional\u0026lt;File\u0026gt; getLargeFile() { return largeFile.get(); } private synchronized Optional\u0026lt;File\u0026gt; createAndCacheFile(String path) { class LargeFileFactory implements Supplier\u0026lt;Optional\u0026lt;File\u0026gt;\u0026gt; { private final File largeFileInstance = new File(path); @Override public Optional\u0026lt;File\u0026gt; get() { return largeFileInstance.exists() ? Optional.of(largeFileInstance) : Optional.empty(); } } if (!(largeFile instanceof LargeFileFactory)) { largeFile = new LargeFileFactory(); } return largeFile.get(); } } } The original LinkedIn graphic is preserved below.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day25/","summary":"Using a implementation of supplier creating and caching a large file (any Class in this case that will initialize a very large object) lazily and thread-safe.","title":"Day 25: Using a implementation of supplier creating and caching a large file (any Class in this case that will initialize a very large object) lazily and thread-safe."},{"content":"Removing nulls from list using Collections.singleton and Objects.isNull\nTranscribed from the original LinkedIn image post.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 import java.util.ArrayList; import java.util.Arrays; import java.util.Collections; import java.util.List; import java.util.Objects; class Day24 { public static void main(String[] args) { String[] filesFromDirUsr = {\u0026#34;A.txt\u0026#34;, \u0026#34;C.txt\u0026#34;, null, \u0026#34;G.txt\u0026#34;}; List\u0026lt;String\u0026gt; usrFiles = new ArrayList\u0026lt;\u0026gt;(Arrays.asList(filesFromDirUsr)); String[] fileFromDirDownloads = {\u0026#34;X.txt\u0026#34;, null, \u0026#34;J.txt\u0026#34;, \u0026#34;N.txt\u0026#34;, null, null}; List\u0026lt;String\u0026gt; downloadedFiles = new ArrayList\u0026lt;\u0026gt;(Arrays.asList(fileFromDirDownloads)); usrFiles.forEach(System.out::println); downloadedFiles.forEach(System.out::println); usrFiles.removeAll(Collections.singleton(null)); downloadedFiles.removeIf(Objects::isNull); System.out.println(); usrFiles.forEach(System.out::println); downloadedFiles.forEach(System.out::println); } } The original LinkedIn graphic is preserved below.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day24/","summary":"Removing nulls from list using Collections.singleton and Objects.isNull","title":"Day 24: Removing nulls from list using Collections.singleton and Objects.isNull"},{"content":"TemporalAdjuster ofDateAdjuster implementation and DayOfWeek calculating the next working day with Bangla name.\nTranscribed from the original LinkedIn image post.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 import java.time.DayOfWeek; import java.time.LocalDate; import java.time.format.TextStyle; import java.time.temporal.TemporalAdjuster; import java.time.temporal.TemporalAdjusters; import java.util.Locale; class Day23 { public static void main(String[] args) { LocalDate today = LocalDate.now(); LocalDate nextWorkingDay = today.with(Day23.nextWorkingDay); DayOfWeek namedDate = nextWorkingDay.getDayOfWeek(); System.out.println(namedDate.getDisplayName(TextStyle.FULL, Locale.forLanguageTag(\u0026#34;bn-BD\u0026#34;))); } static TemporalAdjuster nextWorkingDay = TemporalAdjusters.ofDateAdjuster(localDate -\u0026gt; { DayOfWeek dayOfWeek = localDate.getDayOfWeek(); int daysToAddToGetWorkingDay; int daysToOffsetForFriday = 2; int daysToOffsetForSaturday = 1; int daysToOffsetOtherDay = 1; if (dayOfWeek == DayOfWeek.FRIDAY) daysToAddToGetWorkingDay = daysToOffsetForFriday; else if (dayOfWeek == DayOfWeek.SATURDAY) daysToAddToGetWorkingDay = daysToOffsetForSaturday; else daysToAddToGetWorkingDay = daysToOffsetOtherDay; return localDate.plusDays(daysToAddToGetWorkingDay); }); } The original LinkedIn graphic is preserved below.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day23/","summary":"TemporalAdjuster ofDateAdjuster implementation and DayOfWeek calculating the next working day with Bangla name.","title":"Day 23: TemporalAdjuster ofDateAdjuster implementation and DayOfWeek calculating the next working day with Bangla name."},{"content":"Using stream of streams and flatmapping it to concat multiple streams\nTranscribed from the original LinkedIn image post.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 import java.time.LocalDate; import java.time.format.DateTimeFormatter; import java.util.List; import java.util.function.Function; import java.util.stream.Stream; class Day22 { public static void main(String[] args) { DateTimeFormatter dateTimeFormatter = DateTimeFormatter.ofPattern(\u0026#34;dd/MM/yyyy\u0026#34;); List\u0026lt;String\u0026gt; publicHolidaysOfBD = List.of(\u0026#34;16/02/2021\u0026#34;, \u0026#34;21/02/2021\u0026#34;, \u0026#34;14/04/2021\u0026#34;, \u0026#34;17/03/2021\u0026#34;); List\u0026lt;String\u0026gt; publicHolidaysOfUS = List.of(\u0026#34;01/01/2021\u0026#34;, \u0026#34;17/03/2021\u0026#34;, \u0026#34;04/07/2021\u0026#34;, \u0026#34;14/02/2021\u0026#34;); List\u0026lt;String\u0026gt; publicHolidaysOfCA = List.of(\u0026#34;01/01/2021\u0026#34;, \u0026#34;20/04/2021\u0026#34;, \u0026#34;01/07/2021\u0026#34;, \u0026#34;17/03/2021\u0026#34;); Stream\u0026lt;Stream\u0026lt;String\u0026gt;\u0026gt; streamConcat = Stream.of( publicHolidaysOfCA.stream(), publicHolidaysOfBD.stream(), publicHolidaysOfUS.stream() ); streamConcat .flatMap(Function.identity()) .map(convertToLocalDate(dateTimeFormatter)) .forEach(System.out::println); } static Function\u0026lt;String, LocalDate\u0026gt; convertToLocalDate(DateTimeFormatter dateTimeFormatter) { return (date -\u0026gt; LocalDate.parse(date, dateTimeFormatter)); } } The original LinkedIn graphic is preserved below.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day22/","summary":"Using stream of streams and flatmapping it to concat multiple streams","title":"Day 22: Using stream of streams and flatmapping it to concat multiple streams"},{"content":"Optional.ifPresent and Optional.filter to log and filter the Scheduled Day according to holiday\nTranscribed from the original LinkedIn image post.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 import java.time.LocalDate; import java.util.Optional; import org.slf4j.Logger; import org.slf4j.LoggerFactory; class Day21 { public static void main(String[] args) throws Exception { Logger logger = LoggerFactory.getLogger(Day21.class); Optional\u0026lt;LocalDate\u0026gt; scheduleDay = ScheduleService.getAvailableScheduleDay(); scheduleDay.ifPresent(localDate -\u0026gt; logger.info(\u0026#34;Available Schedule day is {}\u0026#34;, localDate)); LocalDate scheduleDateToString = scheduleDay .filter(HolidayDateService::checkHoliday) .orElseThrow(() -\u0026gt; new NoHolidayException(\u0026#34;No available Sync Time. Reschedule\u0026#34;)); } } The original LinkedIn graphic is preserved below.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day21/","summary":"Optional.ifPresent and Optional.filter to log and filter the Scheduled Day according to holiday","title":"Day 21: Optional.ifPresent and Optional.filter to log and filter the Scheduled Day according to holiday"},{"content":"Some previous post had optional.get() in them which is not a best practice if its not with Optional.isPresent(). better than isPresent is to use orElse(), orElseGet(), orElseThrow()\nTranscribed from the original LinkedIn image post.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 import java.util.OptionalInt; import java.util.function.IntSupplier; import java.util.stream.IntStream; class Day20 { public static void main(String[] args) throws Exception { int defaultNumber = Integer.MAX_VALUE; IntSupplier intSupplier = () -\u0026gt; (int) (Math.random() * 100); IntStream intStream = IntStream.range(0, 10); OptionalInt maxValue = intStream .filter(i -\u0026gt; i \u0026gt; 3) .findAny(); // get value. if not present then get the default number int maxValueOrElse = maxValue.orElse(defaultNumber); // get value. if not present get free provided supplier int maxValueOrElseGet = maxValue.orElseGet(intSupplier); // get value. if not present throws an exception int maxValueOrElseThrow = maxValue.orElseThrow(() -\u0026gt; new Exception(\u0026#34;No value present\u0026#34;)); } } The original LinkedIn graphic is preserved below.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day20/","summary":"Some previous post had optional.get() in them which is not a best practice if its not with Optional.isPresent(). better than isPresent is to use orElse(), orElseGet(), orElseThrow()","title":"Day 20: Some previous post had optional.get() in them which is not a best practice if its not with Optional.isPresent(). better than isPresent is to use orElse(), orElseGet(), orElseThrow()"},{"content":"Stream.peek to get debug information and log it.\nTranscribed from the original LinkedIn image post.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 import java.io.File; import java.io.IOException; import java.nio.file.Files; import java.nio.file.Path; import java.nio.file.attribute.BasicFileAttributes; import java.util.Date; import java.util.List; import org.slf4j.Logger; import org.slf4j.LoggerFactory; class Day19 { public static final Logger logger = LoggerFactory.getLogger(Day19.class); private static void logFileMetadata(File file) { Path path = Path.of(file.getAbsolutePath()); BasicFileAttributes basicFileAttributes = null; try { basicFileAttributes = Files.readAttributes(path, BasicFileAttributes.class); } catch (IOException e) { e.printStackTrace(); } logger.info( \u0026#34;File Name is {}\\n Creation Time {}\\n Size {}\\n Last modified {}\u0026#34;, file.getName(), new Date(basicFileAttributes.creationTime().toMillis()), basicFileAttributes.size() / 1024, new Date(basicFileAttributes.lastModifiedTime().toMillis()) ); } public static void main(String[] args) { List\u0026lt;File\u0026gt; selectedFilesForUpload = getFileList(\u0026#34;/home/mohibulhasan/Downloads\u0026#34;); selectedFilesForUpload.stream() .filter(File::exists) .map(File::getAbsoluteFile) .peek(Day19::logFileMetadata) .forEach(file -\u0026gt; FileUploadService.upload(file)); } } The original LinkedIn graphic is preserved below.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day19/","summary":"Stream.peek to get debug information and log it.","title":"Day 19: Stream.peek to get debug information and log it."},{"content":"using Scanner.findAll to get all the download links from a json file.\nTranscribed from the original LinkedIn image post.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 import java.io.File; import java.io.FileNotFoundException; import java.util.Scanner; import java.util.regex.MatchResult; class Day18 { public static void main(String[] args) { File file = new File(\u0026#34;/home/mohibulhasan/Downloads/data.json\u0026#34;); String urlRegex = \u0026#34;\\\u0026#34;(https?|ftp|file)://[-a-zA-Z0-9+\u0026amp;@#/%?=~_|!:,.;]*[-a-zA-Z0-9+\u0026amp;@#/%=~_|]\\\u0026#34;\u0026#34;; String propertyName = \u0026#34;\\\u0026#34;downloadLink\\\u0026#34;:\\\u0026#34;\u0026#34;; try (Scanner scanner = new Scanner(file)) { scanner.findAll(propertyName + urlRegex) .map(MatchResult::group) .forEach(System.out::println); } catch (FileNotFoundException e) { e.printStackTrace(); } } } The original LinkedIn graphic is preserved below.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day18/","summary":"using Scanner.findAll to get all the download links from a json file.","title":"Day 18: using Scanner.findAll to get all the download links from a json file."},{"content":"Unicode decoding side effect. Application throws exception when a comment is removed.\nTranscribed from the original LinkedIn image post.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 /* /u000d in unicode is new line so when unicode decoding happens the code is in new line and becomes valid java code. Unicode decoding is a process for decoding unicode characters then converting it to ASCII while java code compiles it, first unicode decoding happens then lexical translation */ class Day17 { public static void main(String[] args) { int number = 0; // Please don\u0026#39;t remove below comment. Program will crash // \\u000d number = 10; System.out.println(10 / number); } } The original LinkedIn graphic is preserved below.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day17/","summary":"Unicode decoding side effect. Application throws exception when a comment is removed.","title":"Day 17: Unicode decoding side effect. Application throws exception when a comment is removed."},{"content":"Two interface having same signature default method causing to override them.\nTranscribed from the original LinkedIn image post.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 import java.io.File; import java.io.IOException; import java.nio.file.Files; import java.nio.file.Paths; interface FileUploadExternalDevice { default void upload(File file, String path) throws IOException { Files.move(file.toPath(), Paths.get(path)); } } interface FileUploadAPIEndpoint { default void upload(File file, String path) throws IOException { MultipartFile multiPartFile = multiPart(file); fileUploadService.storeFile(multiPartFile); } } class Day015 implements FileUploadExternalDevice, FileUploadAPIEndpoint { @Override public void upload(File file, String path) throws IOException { /* implementation or FileUploadExternalDevice.super.upload(file, path); or FileUploadAPIEndpoint.super.upload(file, path); */ } } The original LinkedIn graphic is preserved below.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day16/","summary":"Two interface having same signature default method causing to override them.","title":"Day 16: Two interface having same signature default method causing to override them."},{"content":"Batching files and moving them using intStream\nTranscribed from the original LinkedIn image post.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 import java.io.File; import java.io.IOException; import java.nio.file.*; import java.util.ArrayList; import java.util.List; import java.util.stream.Collectors; import java.util.stream.IntStream; import java.util.stream.Stream; class Day015 { public static void main(String[] args) { int batchSize = 100; Path dirPath = getDirPath(); final List\u0026lt;File\u0026gt; files = getFileList(\u0026#34;/home/mohibulhasan/Downloads\u0026#34;); int batchRange = (files.size() + batchSize - 1) / batchSize; IntStream.range(0, batchRange) .mapToObj(i -\u0026gt; files.subList(i * batchSize, Math.min(files.size(), (i + 1) * batchSize))) .forEach(filesInBatch -\u0026gt; moveToDir(filesInBatch, dirPath)); } static List\u0026lt;File\u0026gt; getFileList(String path) { List\u0026lt;File\u0026gt; files = new ArrayList\u0026lt;\u0026gt;(); try (Stream\u0026lt;Path\u0026gt; pathStream = Files.walk(Paths.get(path))) { files = pathStream .filter(Files::isRegularFile) .map(Path::toFile) .collect(Collectors.toList()); } catch (IOException e) { e.printStackTrace(); } return files; } } The original LinkedIn graphic is preserved below.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day15/","summary":"Batching files and moving them using intStream","title":"Day 15: Batching files and moving them using intStream"},{"content":"Using stream parallel to add id to file names.\nTranscribed from the original LinkedIn image post.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 import java.io.IOException; import java.nio.file.*; import java.util.ArrayList; import java.util.List; import java.util.function.Function; import java.util.stream.Collectors; import java.util.stream.Stream; class Day014 { public static void main(String[] args) { Function\u0026lt;String, String\u0026gt; addIdToFile = (file) -\u0026gt; { String hashCode = Integer.toString(file.hashCode()); return file + \u0026#34;-\u0026#34; + hashCode; }; List\u0026lt;String\u0026gt; fileNames = getFileNameList(\u0026#34;/home/mohibulhasan/Downloads\u0026#34;); fileNames.stream() .parallel() .map(addIdToFile) .forEach(System.out::println); } public static List\u0026lt;String\u0026gt; getFileNameList(String path) { Function\u0026lt;String, String\u0026gt; getFileName = (file) -\u0026gt; { int index = file.lastIndexOf(\u0026#34;.\u0026#34;); if (index == -1) return \u0026#34;\u0026#34;; return file.substring(0, index); }; List\u0026lt;String\u0026gt; fileNameList = new ArrayList\u0026lt;\u0026gt;(); try (Stream\u0026lt;Path\u0026gt; paths = Files.walk(Paths.get(path))) { fileNameList = paths .filter(Files::isRegularFile) .map(path1 -\u0026gt; path1.getFileName().toString()) .filter(fileName -\u0026gt; !fileName.isEmpty()) .map(getFileName) .collect(Collectors.toList()); } catch (IOException e) { System.out.println(\u0026#34;Exception Occured \u0026#34; + e.getMessage()); } return fileNameList; } } The original LinkedIn graphic is preserved below.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day14/","summary":"Using stream parallel to add id to file names.","title":"Day 14: Using stream parallel to add id to file names."},{"content":"Using Collectors.groupingBy and Collectors.counting to get numbers of file present of a particular type\nTranscribed from the original LinkedIn image post.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 import java.io.IOException; import java.nio.file.*; import java.util.Map; import java.util.function.Function; import java.util.stream.Collectors; import java.util.stream.Stream; class Scratch { public static void main(String[] args) throws IOException, InterruptedException { Function\u0026lt;String, String\u0026gt; fileNameRemove = (fileName) -\u0026gt; { int index = fileName.lastIndexOf(\u0026#34;.\u0026#34;); if (index == -1) return \u0026#34;\u0026#34;; return fileName.substring(index); }; try (Stream\u0026lt;Path\u0026gt; paths = Files.walk(Paths.get(\u0026#34;/home/mohibulhasan/Downloads\u0026#34;))) { Map\u0026lt;String, Long\u0026gt; fileCountByType = paths .filter(path -\u0026gt; Files.isRegularFile(path)) .map(path -\u0026gt; path.getFileName().toString()) .filter(fileName -\u0026gt; !fileName.isEmpty()) .map(fileNameRemove) .collect(Collectors.groupingBy( Function.identity(), Collectors.counting() )); System.out.println(fileCountByType); } } } The original LinkedIn graphic is preserved below.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day13/","summary":"Using Collectors.groupingBy and Collectors.counting to get numbers of file present of a particular type","title":"Day 13: Using Collectors.groupingBy and Collectors.counting to get numbers of file present of a particular type"},{"content":"Files.walk() and functional interface to remove file extension from fileNames\nTranscribed from the original LinkedIn image post.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 import java.io.IOException; import java.nio.file.*; import java.util.List; import java.util.function.Function; import java.util.stream.Collectors; import java.util.stream.Stream; class Day12 { public static void main(String[] args) throws IOException, InterruptedException { Function\u0026lt;String, String\u0026gt; extensionRemove = (fileName) -\u0026gt; { int index = fileName.lastIndexOf(\u0026#34;.\u0026#34;); if (index == -1) return \u0026#34;\u0026#34;; return fileName.substring(0, index); }; List\u0026lt;String\u0026gt; fileNameList; try (Stream\u0026lt;Path\u0026gt; paths = Files.walk(Paths.get(\u0026#34;/home/mohibulhasan/Downloads\u0026#34;))) { fileNameList = paths .filter(Files::isRegularFile) .map(path -\u0026gt; path.getFileName().toString()) .filter(fileName -\u0026gt; !fileName.isEmpty()) .map(extensionRemove) .collect(Collectors.toList()); } fileNameList.forEach(System.out::println); } } The original LinkedIn graphic is preserved below.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day12/","summary":"Files.walk() and functional interface to remove file extension from fileNames","title":"Day 12: Files.walk() and functional interface to remove file extension from fileNames"},{"content":"Supplier, Consumer functional interface and httpclient to get and output Bitcoin info\nTranscribed from the original LinkedIn image post.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 import java.io.IOException; import java.net.URI; import java.net.http.HttpClient; import java.net.http.HttpRequest; import java.net.http.HttpResponse; import java.util.function.Consumer; import java.util.function.Supplier; class Day11 { public static void main(String[] args) throws IOException, InterruptedException { Supplier\u0026lt;String\u0026gt; jsonStringSupplier = () -\u0026gt; getBitcoinInfoAsString(); String str = jsonStringSupplier.get(); Consumer\u0026lt;String\u0026gt; consumeString = s -\u0026gt; System.out.println(s); consumeString.accept(str); } public static String getBitcoinInfoAsString() { HttpClient httpClient = HttpClient.newHttpClient(); HttpRequest httpRequest = HttpRequest .newBuilder() .uri(URI.create(\u0026#34;https://api.coindesk.com/v1/bpi/currentprice.json\u0026#34;)) .header(\u0026#34;Accept\u0026#34;, \u0026#34;application/json\u0026#34;) .build(); HttpResponse\u0026lt;String\u0026gt; httpResponse = null; try { httpResponse = httpClient.send(httpRequest, HttpResponse.BodyHandlers.ofString()); } catch (IOException e) { e.printStackTrace(); } catch (InterruptedException e) { e.printStackTrace(); } return httpResponse.body().toString(); } } The original LinkedIn graphic is preserved below.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day11/","summary":"Supplier, Consumer functional interface and httpclient to get and output Bitcoin info","title":"Day 11: Supplier, Consumer functional interface and httpclient to get and output Bitcoin info"},{"content":"Function is a functional interface that takes an input and gives an output. so functions can be chained.\nTranscribed from the original LinkedIn image post.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 import java.io.IOException; import java.util.function.Function; public class Day10 { public static void main(String[] args) throws IOException, InterruptedException { Function\u0026lt;String, String\u0026gt; stringFunction = String::trim; Function\u0026lt;String, Boolean\u0026gt; stringFunctionChar = stringFunction .andThen(String::toLowerCase) .andThen(s -\u0026gt; s.contains(\u0026#34;function\u0026#34;)); Boolean check = stringFunctionChar.apply(\u0026#34;A string to put in function\u0026#34;); System.out.println(check.booleanValue()); } } The original LinkedIn graphic is preserved below.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day10/","summary":"Function is a functional interface that takes an input and gives an output. so functions can be chained.","title":"Day 10: Function is a functional interface that takes an input and gives an output. so functions can be chained."},{"content":"Functional interface\nA functional interface or single abstract method interface is a special type of interface. It can still declare default methods, and even toString() without breaking the single-abstract-method constraint because the implementation can come from Object at runtime.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 @FunctionalInterface interface MyFunctionalInterface { void init(); default String addString(String str) { return str + str; } String toString(); } public class Day09 { public static void main(String[] args) throws IOException, InterruptedException { MyFunctionalInterface myFunctionalInterface = () -\u0026gt; System.out.println(\u0026#34;My functional Interface\u0026#34;); myFunctionalInterface.init(); } } The original LinkedIn graphic is preserved below.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day9/","summary":"Functional interface","title":"Day 9: Functional interface"},{"content":"I always liked string interpolation in JS. Now i found out about Javas MessageFormat\u0026rsquo;\u0026rsquo;s format method which lets us in its way use string interpolation.\nTranscribed from the original LinkedIn image post.\n1 2 3 4 5 6 7 8 9 10 11 12 13 import java.text.MessageFormat; public class Day08 { public static void main(String[] args) { String stringInterpolation = MessageFormat .format( \u0026#34;MessageFormat.format makes {0} {1} happen\u0026#34;, \u0026#34;string\u0026#34;, \u0026#34;interpolation\u0026#34; ); System.out.println(stringInterpolation); } } The original LinkedIn graphic is preserved below.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day8/","summary":"I always liked string interpolation in JS. Now i found out about Javas MessageFormat\u0026rsquo;s format method which lets us in its way use string interpolation.","title":"Day 8: I always liked string interpolation in JS. Now i found out about Javas MessageFormat's format method which lets us in its way use string interpolation."},{"content":"Today i found out that string declared with \u0026quot;\u0026quot; are placed in string constant pool while with new keyword its in heap. String.intern() return string if defined in string pool otherwise creates the string and then returns it.\nTranscribed from the original LinkedIn image post.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 public class Day07 { public static void main(String args[]) { // String is created in heap String file1 = new String(\u0026#34;docx\u0026#34;); String file2 = new String(\u0026#34;docx\u0026#34;); // checking for memory location System.out.println(file1 == file2); // false // String is created in String constant pool String file3 = \u0026#34;rs\u0026#34;; String file4 = \u0026#34;rs\u0026#34;; System.out.println(file3 == file4); // true // now using intern() getting a copy of docx in String pool String file5 = file2.intern(); String file6 = \u0026#34;docx\u0026#34;; // first check file2 and file5 System.out.println(file2 == file5); // next check file5 and file6 System.out.println(file5 == file6); } } The original LinkedIn graphic is preserved below.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day7/","summary":"Today i found out that string declared with \u0026quot;\u0026quot; are placed in string constant pool while with new keyword its in heap. String.intern() return string if defined in string pool otherwise creates the string and then returns it.","title":"Day 7: Today i found out that string declared with \"\" are placed in string constant pool while with new keyword its in heap. String.intern() return string if defined in string pool otherwise creates the string and then returns it."},{"content":" Update, May 2026: the original benchmark below should not be treated as evidence that a classic for loop is meaningfully faster than foreach. It used System.currentTimeMillis(), no warmup, no repeated forks, and loops whose work could be optimized away. A proper Java microbenchmark should use JMH and should consume real work so the JIT cannot remove the measurement target.\nThis post is kept as a learning snapshot, but the conclusion has changed: prefer the loop that makes the code clearer unless a production measurement with representative data shows otherwise.\nTranscribed from the original LinkedIn image post.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 import java.util.ArrayList; import java.util.List; public class Day06 { public static void main(String args[]) { List\u0026lt;String\u0026gt; fileTypeList = new ArrayList\u0026lt;\u0026gt;(); for (int i = 0; i \u0026lt; 1000000; i++) { fileTypeList.add(\u0026#34;fileType\u0026#34;); } long beforeForLoop = System.currentTimeMillis(); for (int i = 0; i \u0026lt; fileTypeList.size(); i++) { fileTypeList.get(i); } long afterForLoop = System.currentTimeMillis(); System.out.println(\u0026#34;Time took in millis for for \u0026#34; + (afterForLoop - beforeForLoop)); long beforeForeachLoop = System.currentTimeMillis(); for (String s : fileTypeList) { } long afterForeachLoop = System.currentTimeMillis(); System.out.println(\u0026#34;Time took in millis for foreach \u0026#34; + (afterForeachLoop - beforeForeachLoop)); } } What a better benchmark needs A useful version of this test needs:\nJMH warmup and measurement iterations. Several forks so one JVM run does not dominate the result. Real work inside each loop. A consumed result so the JIT cannot remove the loop. Representative data structures, because arrays, ArrayList, LinkedList, and custom collections do not behave the same way. For this series, Day 69 covers JMH in more detail: Day 69 - Unlocking Java Performance Secrets.\nThe original LinkedIn graphic is preserved below.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day6/","summary":"The original quick timing test was a learning snapshot, not evidence that for loops are faster than foreach loops. This update explains the benchmark flaws and points to JMH.","title":"Day 6: For Loop vs Foreach Loop Performance, and Why My First Benchmark Was Misleading"},{"content":"Getting stream of char from string.chars()\nTranscribed from the original LinkedIn image post.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 /** * Getting a stream of characters from String.chars() */ public class Day05 { public static void main(String[] args) { String fileNameWithExtension = \u0026#34;a-very-large-file-name.docx\u0026#34;; // this will give an IntStream where the int\u0026#39;s are character code // Getting characters by mapping to char fileNameWithExtension.chars() .mapToObj(ch -\u0026gt; (char) ch) .forEach(System.out::println); } } The original LinkedIn graphic is preserved below.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day5/","summary":"Getting stream of char from string.chars()","title":"Day 5: Getting stream of char from string.chars()"},{"content":"try with resource helps to close open resources that uses .close() method to close.\nTranscribed from the original LinkedIn image post.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 /* Try with resource is basically closing the resources where a programmer would have to manually close FileWriter and BufferedWriter */ import java.io.BufferedWriter; import java.io.File; import java.io.FileWriter; import java.io.IOException; public class Day04 { public static void main(String[] args) throws IOException { File file = new File(\u0026#34;/home/mohibulhasan/Documents/example.txt\u0026#34;); // try with resource try ( FileWriter fileWriter = new FileWriter(file); BufferedWriter bufferedWriter = new BufferedWriter(fileWriter) ) { bufferedWriter.write(\u0026#34;Found about try with resource.\u0026#34;); bufferedWriter.newLine(); } } } The original LinkedIn graphic is preserved below.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day4/","summary":"try with resource helps to close open resources that uses .close() method to close.","title":"Day 4: try with resource helps to close open resources that uses .close() method to close."},{"content":"Using Objects.requireNonNullElse() where Optional not being able to be used.\nTranscribed from the original LinkedIn image post.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 /* When returning a object that can have null value. wrapping it with an optional can give you NullPointerException if object contains null. In this case using optional might create some issue. */ import java.util.*; public class Day03 { public static void main(String[] args) { // String fileExtension = getFileExtensionOpt().get(); String fileExtension = Objects.requireNonNullElse( getFileExtension(), \u0026#34;got null in getFileExtension()\u0026#34; ); System.out.println(fileExtension); } // will cause NullPointerException when get() is called on returned object public static Optional\u0026lt;String\u0026gt; getFileExtensionOpt() { return Optional.of(null); } // without optional public static String getFileExtension() { return null; } } The original LinkedIn graphic is preserved below.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day3/","summary":"Using Objects.requireNonNullElse() where Optional not being able to be used.","title":"Day 3: Using Objects.requireNonNullElse() where Optional not being able to be used."},{"content":"Getting the last element of a Stream.\nTranscribed from the original LinkedIn image post.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 /* Getting the last element of a stream */ import java.util.List; import java.util.stream.*; public class Day02 { public static void main(String[] args) { List\u0026lt;String\u0026gt; fileTypeList = List.of(\u0026#34;jpg\u0026#34;, \u0026#34;png\u0026#34;, \u0026#34;avi\u0026#34;, \u0026#34;mpeg\u0026#34;, \u0026#34;docx\u0026#34;); // using reduce String lastElement = fileTypeList.stream() .reduce((element1, element2) -\u0026gt; element2) .get(); System.out.println(lastElement); // using skip String lastElement1 = fileTypeList.stream() .skip(fileTypeList.size() - 1) .findFirst() .get(); System.out.println(lastElement1); } } The original LinkedIn graphic is preserved below.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day2/","summary":"Getting the last element of a Stream.","title":"Day 2: Getting the last element of a Stream."},{"content":"This example checks whether one string is equal to any value from a fixed set of allowed strings.\nTranscribed from the original LinkedIn image post.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 /* Checking equality against multiple string */ import java.util.List; import java.util.stream.*; public class Day01 { public static void main(String[] args) { String fileType = \u0026#34;mpeg\u0026#34;; // have to assure that given fileType and comparing strings // are all in same case (case-sensitiveness) if (List.of(\u0026#34;jpg\u0026#34;, \u0026#34;png\u0026#34;, \u0026#34;avi\u0026#34;, \u0026#34;mpeg\u0026#34;, \u0026#34;docx\u0026#34;).contains(fileType)) { System.out.println(\u0026#34;Founded file with type \u0026#34; + fileType); } // another way of doing it if (Stream.of(\u0026#34;jpg\u0026#34;, \u0026#34;png\u0026#34;, \u0026#34;avi\u0026#34;, \u0026#34;mpeg\u0026#34;, \u0026#34;docx\u0026#34;).anyMatch(fileType::equalsIgnoreCase)) { System.out.println(\u0026#34;Founded file with type \u0026#34; + fileType); } } } Which option should you use? For a small fixed list, List.of(...).contains(value) is simple and readable.\nFor repeated lookups or a larger set, keep the allowed values in a Set:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 import java.util.Locale; import java.util.Set; public class Day01 { private static final Set\u0026lt;String\u0026gt; ALLOWED_FILE_TYPES = Set.of(\u0026#34;jpg\u0026#34;, \u0026#34;png\u0026#34;, \u0026#34;avi\u0026#34;, \u0026#34;mpeg\u0026#34;, \u0026#34;docx\u0026#34;); public static void main(String[] args) { String fileType = \u0026#34;mpeg\u0026#34;; if (ALLOWED_FILE_TYPES.contains(fileType.toLowerCase(Locale.ROOT))) { System.out.println(\u0026#34;Found file with type \u0026#34; + fileType); } } } If a project already uses Apache Commons Lang, Strings.CS.equalsAny(...) or the relevant case-insensitive variant can express this directly without creating a collection at the call site. Streams are better when this check is part of a longer pipeline, not just for a standalone membership test.\nThe original LinkedIn graphic is preserved below.\n","permalink":"https://mohibul.pages.dev/posts/java/100daysofjava/day1/","summary":"A small Java example for checking whether a string matches one of several allowed values, with notes on List.of, Set.of, Apache Commons Lang, and streams.","title":"Day 1: Checking Whether a String Matches One of Several Values"},{"content":"I\u0026rsquo;m a Senior Software Engineer II with 7+ years of experience building backend systems for US and European clients. My work centers on Java, distributed systems, production AI, and reliability.\nI like the part of engineering where a tidy abstraction meets an untidy production system. That has led me to work on call transcription and conversation intelligence, database migrations, payment systems, and failures that cross Java services, Erlang state machines, SQL, and messaging workflows.\nWork I can discuss publicly I helped architect and ship call transcription and conversation-intelligence workflows using Google Vertex AI and Gemini. I led a database cleanup after analysis of approximately 68 million records and cross-system dependencies. The migration removed 8 write triggers and 4 unused tables. I contribute technical writing through Baeldung and maintain the 100DaysOfJava archive and Java Learning Atlas. Read the case studies for scope, evidence, and limitations.\nCareer Cefalo, 2022 to present. Senior Software Engineer II working across Java, Erlang, AI, telephony, databases, payments, and reliability. Brain Station 23, 2020 to 2022. Software Engineer building Spring Boot APIs, tenant-aware authorization, Stripe billing, and healthcare-platform features. Welldev, 2019 to 2020. Junior Software Engineer working with Spring Boot, PostgreSQL, Vue.js, Nuxt.js, Storybook, and Docker. Writing I write investigations that start with a concrete question, show the evidence, and state what the evidence cannot prove. Browse the technical writing or use the Java Learning Atlas to follow the 100DaysOfJava series by topic.\nWork and contact I welcome conversations about senior engineering roles, backend and AI architecture, reliability work, and focused consulting engagements.\nDownload my resume View my resume page Email me LinkedIn GitHub I also publish Chaos;Code;Clarity, a newsletter about software engineering and AI.\n","permalink":"https://mohibul.pages.dev/about/","summary":"Senior Software Engineer II working with Java, distributed systems, production AI, and reliability.","title":"About Mohibul Hassan Chowdhury"},{"content":"Choose a path into the archive:\nDebug a Java service Understand concurrency Build reliable AI workflows The collection has two Day 100 entries. The Day 100 retrospective looks back at the full project. The Spliterator investigation is the final performance experiment.\n","permalink":"https://mohibul.pages.dev/java/100daysofjava/graph/","summary":"\u003cp\u003eChoose a path into the archive:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"/posts/java/100daysofjava/day66/\"\u003eDebug a Java service\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"/posts/posts/java/100daysofjava/day99/\"\u003eUnderstand concurrency\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"/posts/posts/java/100daysofjava/day88/\"\u003eBuild reliable AI workflows\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThe collection has two Day 100 entries. The \u003ca href=\"/posts/posts/java/100daysofjava/day100-capstone.md/\"\u003eDay 100 retrospective\u003c/a\u003e looks back at the full project. The \u003ca href=\"/posts/posts/java/100daysofjava/day100-spliterator/\"\u003eSpliterator investigation\u003c/a\u003e is the final performance experiment.\u003c/p\u003e","title":"Java Learning Atlas"},{"content":"I\u0026rsquo;m a Senior Software Engineer II with 7+ years of experience building backend systems, distributed services, and production AI workflows.\nDownload the PDF or read the Selected work behind the summary.\nCurrent focus Java backend and distributed systems Production AI, transcription, and conversation intelligence Database migrations, performance, and reliability Contact Email LinkedIn GitHub ","permalink":"https://mohibul.pages.dev/resume/","summary":"Resume and contact details for Mohibul Hassan Chowdhury.","title":"Resume"},{"content":"These examples separate production responsibilities from public technical investigations. Customer names and private data are omitted.\nProduction call intelligence Problem. Turn call recordings into useful transcripts and conversation intelligence while accounting for language, model uncertainty, retries, and later transcript revisions.\nConstraints. The workflows needed single-recording and batch processing, multilingual prompting, credential management, confidence-based evaluation, and failure handling across backend and administration flows. Model reliability and bring-your-own-key pricing also affected the design.\nMy contribution. I worked on the architecture and implementation of transcription workflows using Google Vertex AI and Gemini. I also worked on transcript-derived summaries whose state becomes stale when a transcript changes.\nDecisions and evidence. The workflow records append-only lifecycle state, supports batch processing and retries, uses feature flags for controlled release, and sends notifications when work completes. Confidence signals and provider behavior inform evaluation instead of treating every model response as equally reliable.\nOutcome. The system supports production transcription and conversation-intelligence workflows without tying the backend to one happy-path request.\nLimitations. I cannot publish customer data, private evaluation sets, traffic volumes, or internal reliability measurements.\nDatabase cleanup and migration Problem. A mature platform was carrying database structures and recurring writes whose current value needed to be established before removal.\nConstraints. The analysis covered approximately 68 million records and dependencies across multiple systems. Removing data structures required careful sequencing, validation, and rollback safeguards.\nMy contribution. I traced dependencies, helped define the migration sequence, and verified the effect of removing 8 write triggers and 4 unused tables.\nDecisions and evidence. The migration separated dependency analysis from removal, retained rollback options until validation passed, and measured database activity after the change.\nOutcome. The cleanup eliminated approximately 1.8 million unnecessary writes per month. It reclaimed an estimated 50 to 100 GB on affected database instances and contributed to an estimated USD 600 to 1,200 in annual infrastructure savings.\nLimitations. Storage and cost figures are estimates. This case study does not claim that an outage or a single defect caused the accumulated data.\nVirtual threads and event loops Technical investigation. I built two small HTTP servers and a benchmark suite to compare virtual threads with an event-loop design. The article records the machine details, commands, results, and limits of the sample run.\nRead Virtual threads and event loops solve different problems and inspect the public benchmark repository.\nThis is an educational implementation, not evidence that the benchmark architecture ran in production.\nContact Email LinkedIn Download the resume ","permalink":"https://mohibul.pages.dev/case-studies/","summary":"Selected production work and technical investigations by Mohibul Hassan Chowdhury.","title":"Selected work"},{"content":" Support my work\nMohibul Hassan\nSoftware Engineer and Technical Writer\nI write about software engineering, Java, JavaScript, cloud platforms, and AI. If the writing or projects here helped you, you can support the work or start a professional conversation. SupportKori Buy Me a Coffee nSave Email Me Local Payments International bK Mobile payment\nbKash Personal account. The phone number is intentionally not published.\nQR code coming soon\nQR pending BB Bank transfer\nBRAC Bank PLC Use these details for local bank transfers in Bangladesh.\nCopy all bank details Account Name MOHIBUL HASSAN CHOWDHURY Copy Account Number 1043831890002 Copy Branch Name SATMASJID ROAD BRANCH Copy Routing Number 060276074 Copy SWIFT Code BRAKBDDH Copy S Local support\nSupportKori Use this for local one-time support.\nsupportkori.com/mohibulhassan\nSupport now Copy link QR B Direct support\nBuy Me a Coffee Use this for one-time support for articles, Java notes, and side projects.\nBuy a coffee Copy link QR ns International support\nnSave Use this tag or link for international support.\n@chowdhu_mohibul\nOpen nSave Copy tag Copy link QR M Professional contact\nEmail For consulting, writing, collaboration, invoices, or private payment details.\nmohibulhassan100@gmail.com\nSend email Copy email Profiles G Open source\nGitHub Review the site source, projects, and experiments I share publicly.\ngithub.com/dhrubo55\nVisit GitHub Copy link in Network\nLinkedIn Connect for engineering roles, technical writing work, and professional updates.\nlinkedin.com/in/mohibulhassan\nOpen LinkedIn Copy link Wise and WhatsApp are intentionally not listed. bKash number is private for now; I will add the QR code when it is ready. x Scan QR\nQR code\n","permalink":"https://mohibul.pages.dev/support/","summary":"Support Mohibul Hassan\u0026rsquo;s writing and software engineering projects.","title":"Support Mohibul Hassan"}]